Do functional status and Medicare claims data improve the predictive accuracy of an electronic health record mortality index? Findings from a national Veterans Affairs cohort
Abstract Background Electronic health record (EHR) prediction models may be easier to use in busy clinical settings since EHR data can be auto-populated into models. This study assessed whether adding functional status and/or Medicare claims data (which are often not available in EHRs) improves the...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-05-01
|
| Series: | BMC Geriatrics |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12877-022-03126-z |
| _version_ | 1818238815918096384 |
|---|---|
| author | William James Deardorff Bocheng Jing Sun Y. Jeon W. John Boscardin Alexandra K. Lee Kathy Z. Fung Sei J. Lee |
| author_facet | William James Deardorff Bocheng Jing Sun Y. Jeon W. John Boscardin Alexandra K. Lee Kathy Z. Fung Sei J. Lee |
| author_sort | William James Deardorff |
| collection | DOAJ |
| description | Abstract Background Electronic health record (EHR) prediction models may be easier to use in busy clinical settings since EHR data can be auto-populated into models. This study assessed whether adding functional status and/or Medicare claims data (which are often not available in EHRs) improves the accuracy of a previously developed Veterans Affairs (VA) EHR-based mortality index. Methods This was a retrospective cohort study of veterans aged 75 years and older enrolled in VA primary care clinics followed from January 2014 to April 2020 (n = 62,014). We randomly split participants into development (n = 49,612) and validation (n = 12,402) cohorts. The primary outcome was all-cause mortality. We performed logistic regression with backward stepwise selection to develop a 100-predictor base model using 854 EHR candidate variables, including demographics, laboratory values, medications, healthcare utilization, diagnosis codes, and vitals. We incorporated functional measures in a base + function model by adding activities of daily living (range 0-5) and instrumental activities of daily living (range 0-7) scores. Medicare data, including healthcare utilization (e.g., emergency department visits, hospitalizations) and diagnosis codes, were incorporated in a base + Medicare model. A base + function + Medicare model included all data elements. We assessed model performance with the c-statistic, reclassification metrics, fraction of new information provided, and calibration plots. Results In the overall cohort, mean age was 82.6 years and 98.6% were male. At the end of follow-up, 30,263 participants (48.8%) had died. The base model c-statistic was 0.809 (95% CI 0.805-0.812) in the development cohort and 0.804 (95% CI 0.796-0.812) in the validation cohort. Validation cohort c-statistics for the base + function, base + Medicare, and base + function + Medicare models were 0.809 (95% CI 0.801-0.816), 0.811 (95% CI 0.803-0.818), and 0.814 (95% CI 0.807-0.822), respectively. Adding functional status and Medicare data resulted in similarly small improvements among other model performance measures. All models showed excellent calibration. Conclusions Incorporation of functional status and Medicare data into a VA EHR-based mortality index led to small but likely clinically insignificant improvements in model performance. |
| first_indexed | 2024-12-12T12:47:39Z |
| format | Article |
| id | doaj.art-ed03503a99874d90af66c8731290287e |
| institution | Directory Open Access Journal |
| issn | 1471-2318 |
| language | English |
| last_indexed | 2024-12-12T12:47:39Z |
| publishDate | 2022-05-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Geriatrics |
| spelling | doaj.art-ed03503a99874d90af66c8731290287e2022-12-22T00:24:05ZengBMCBMC Geriatrics1471-23182022-05-012211910.1186/s12877-022-03126-zDo functional status and Medicare claims data improve the predictive accuracy of an electronic health record mortality index? Findings from a national Veterans Affairs cohortWilliam James Deardorff0Bocheng Jing1Sun Y. Jeon2W. John Boscardin3Alexandra K. Lee4Kathy Z. Fung5Sei J. Lee6Division of Geriatrics, University of California, San FranciscoDivision of Geriatrics, University of California, San FranciscoDivision of Geriatrics, University of California, San FranciscoDivision of Geriatrics, University of California, San FranciscoDivision of Geriatrics, University of California, San FranciscoGeriatrics, Palliative and Extended Care Service Line, San Francisco Veterans Affairs Health Care SystemGeriatrics, Palliative and Extended Care Service Line, San Francisco Veterans Affairs Health Care SystemAbstract Background Electronic health record (EHR) prediction models may be easier to use in busy clinical settings since EHR data can be auto-populated into models. This study assessed whether adding functional status and/or Medicare claims data (which are often not available in EHRs) improves the accuracy of a previously developed Veterans Affairs (VA) EHR-based mortality index. Methods This was a retrospective cohort study of veterans aged 75 years and older enrolled in VA primary care clinics followed from January 2014 to April 2020 (n = 62,014). We randomly split participants into development (n = 49,612) and validation (n = 12,402) cohorts. The primary outcome was all-cause mortality. We performed logistic regression with backward stepwise selection to develop a 100-predictor base model using 854 EHR candidate variables, including demographics, laboratory values, medications, healthcare utilization, diagnosis codes, and vitals. We incorporated functional measures in a base + function model by adding activities of daily living (range 0-5) and instrumental activities of daily living (range 0-7) scores. Medicare data, including healthcare utilization (e.g., emergency department visits, hospitalizations) and diagnosis codes, were incorporated in a base + Medicare model. A base + function + Medicare model included all data elements. We assessed model performance with the c-statistic, reclassification metrics, fraction of new information provided, and calibration plots. Results In the overall cohort, mean age was 82.6 years and 98.6% were male. At the end of follow-up, 30,263 participants (48.8%) had died. The base model c-statistic was 0.809 (95% CI 0.805-0.812) in the development cohort and 0.804 (95% CI 0.796-0.812) in the validation cohort. Validation cohort c-statistics for the base + function, base + Medicare, and base + function + Medicare models were 0.809 (95% CI 0.801-0.816), 0.811 (95% CI 0.803-0.818), and 0.814 (95% CI 0.807-0.822), respectively. Adding functional status and Medicare data resulted in similarly small improvements among other model performance measures. All models showed excellent calibration. Conclusions Incorporation of functional status and Medicare data into a VA EHR-based mortality index led to small but likely clinically insignificant improvements in model performance.https://doi.org/10.1186/s12877-022-03126-zFunctional statusPhysical functionMedicare dataMortality prediction model |
| spellingShingle | William James Deardorff Bocheng Jing Sun Y. Jeon W. John Boscardin Alexandra K. Lee Kathy Z. Fung Sei J. Lee Do functional status and Medicare claims data improve the predictive accuracy of an electronic health record mortality index? Findings from a national Veterans Affairs cohort BMC Geriatrics Functional status Physical function Medicare data Mortality prediction model |
| title | Do functional status and Medicare claims data improve the predictive accuracy of an electronic health record mortality index? Findings from a national Veterans Affairs cohort |
| title_full | Do functional status and Medicare claims data improve the predictive accuracy of an electronic health record mortality index? Findings from a national Veterans Affairs cohort |
| title_fullStr | Do functional status and Medicare claims data improve the predictive accuracy of an electronic health record mortality index? Findings from a national Veterans Affairs cohort |
| title_full_unstemmed | Do functional status and Medicare claims data improve the predictive accuracy of an electronic health record mortality index? Findings from a national Veterans Affairs cohort |
| title_short | Do functional status and Medicare claims data improve the predictive accuracy of an electronic health record mortality index? Findings from a national Veterans Affairs cohort |
| title_sort | do functional status and medicare claims data improve the predictive accuracy of an electronic health record mortality index findings from a national veterans affairs cohort |
| topic | Functional status Physical function Medicare data Mortality prediction model |
| url | https://doi.org/10.1186/s12877-022-03126-z |
| work_keys_str_mv | AT williamjamesdeardorff dofunctionalstatusandmedicareclaimsdataimprovethepredictiveaccuracyofanelectronichealthrecordmortalityindexfindingsfromanationalveteransaffairscohort AT bochengjing dofunctionalstatusandmedicareclaimsdataimprovethepredictiveaccuracyofanelectronichealthrecordmortalityindexfindingsfromanationalveteransaffairscohort AT sunyjeon dofunctionalstatusandmedicareclaimsdataimprovethepredictiveaccuracyofanelectronichealthrecordmortalityindexfindingsfromanationalveteransaffairscohort AT wjohnboscardin dofunctionalstatusandmedicareclaimsdataimprovethepredictiveaccuracyofanelectronichealthrecordmortalityindexfindingsfromanationalveteransaffairscohort AT alexandraklee dofunctionalstatusandmedicareclaimsdataimprovethepredictiveaccuracyofanelectronichealthrecordmortalityindexfindingsfromanationalveteransaffairscohort AT kathyzfung dofunctionalstatusandmedicareclaimsdataimprovethepredictiveaccuracyofanelectronichealthrecordmortalityindexfindingsfromanationalveteransaffairscohort AT seijlee dofunctionalstatusandmedicareclaimsdataimprovethepredictiveaccuracyofanelectronichealthrecordmortalityindexfindingsfromanationalveteransaffairscohort |