The impact of electronic health record discontinuity on prediction modeling.
<h4>Background</h4>To determine the impact of electronic health record (EHR)-discontinuity on the performance of prediction models.<h4>Methods</h4>The study population consisted of patients with a history of cardiovascular (CV) comorbidities identified using US Medicare claim...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2023-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0287985 |
| _version_ | 1797774765616791552 |
|---|---|
| author | Shreyas Kar Lily G Bessette Richard Wyss Aaron S Kesselheim Kueiyu Joshua Lin |
| author_facet | Shreyas Kar Lily G Bessette Richard Wyss Aaron S Kesselheim Kueiyu Joshua Lin |
| author_sort | Shreyas Kar |
| collection | DOAJ |
| description | <h4>Background</h4>To determine the impact of electronic health record (EHR)-discontinuity on the performance of prediction models.<h4>Methods</h4>The study population consisted of patients with a history of cardiovascular (CV) comorbidities identified using US Medicare claims data from 2007 to 2017, linked to EHR from two networks (used as model training and validation set, respectively). We built models predicting one-year risk of mortality, major CV events, and major bleeding events, stratified by high vs. low algorithm-predicted EHR-continuity. The best-performing models for each outcome were chosen among 5 commonly used machine-learning models. We compared model performance by Area under the ROC curve (AUROC) and Area under the precision-recall curve (AUPRC).<h4>Results</h4>Based on 180,950 in the training and 103,061 in the validation set, we found EHR captured only 21.0-28.1% of all the non-fatal outcomes in the low EHR-continuity cohort but 55.4-66.1% of that in the high EHR-continuity cohort. In the validation set, the best-performing model developed among high EHR-continuity patients had consistently higher AUROC than that based on low-continuity patients: AUROC was 0.849 vs. 0.743 when predicting mortality; AUROC was 0.802 vs. 0.659 predicting the CV events; AUROC was 0.635 vs. 0.567 predicting major bleeding. We observed a similar pattern when using AUPRC as the outcome metric.<h4>Conclusions</h4>Among patients with CV comorbidities, when predicting mortality, major CV events, and bleeding outcomes, the prediction models developed in datasets with low EHR-continuity consistently had worse performance compared to models developed with high EHR-continuity. |
| first_indexed | 2024-03-12T22:25:59Z |
| format | Article |
| id | doaj.art-af709c02abb44c20bc8de9dcf6401f4c |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-03-12T22:25:59Z |
| publishDate | 2023-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-af709c02abb44c20bc8de9dcf6401f4c2023-07-22T05:31:41ZengPublic Library of Science (PLoS)PLoS ONE1932-62032023-01-01187e028798510.1371/journal.pone.0287985The impact of electronic health record discontinuity on prediction modeling.Shreyas KarLily G BessetteRichard WyssAaron S KesselheimKueiyu Joshua Lin<h4>Background</h4>To determine the impact of electronic health record (EHR)-discontinuity on the performance of prediction models.<h4>Methods</h4>The study population consisted of patients with a history of cardiovascular (CV) comorbidities identified using US Medicare claims data from 2007 to 2017, linked to EHR from two networks (used as model training and validation set, respectively). We built models predicting one-year risk of mortality, major CV events, and major bleeding events, stratified by high vs. low algorithm-predicted EHR-continuity. The best-performing models for each outcome were chosen among 5 commonly used machine-learning models. We compared model performance by Area under the ROC curve (AUROC) and Area under the precision-recall curve (AUPRC).<h4>Results</h4>Based on 180,950 in the training and 103,061 in the validation set, we found EHR captured only 21.0-28.1% of all the non-fatal outcomes in the low EHR-continuity cohort but 55.4-66.1% of that in the high EHR-continuity cohort. In the validation set, the best-performing model developed among high EHR-continuity patients had consistently higher AUROC than that based on low-continuity patients: AUROC was 0.849 vs. 0.743 when predicting mortality; AUROC was 0.802 vs. 0.659 predicting the CV events; AUROC was 0.635 vs. 0.567 predicting major bleeding. We observed a similar pattern when using AUPRC as the outcome metric.<h4>Conclusions</h4>Among patients with CV comorbidities, when predicting mortality, major CV events, and bleeding outcomes, the prediction models developed in datasets with low EHR-continuity consistently had worse performance compared to models developed with high EHR-continuity.https://doi.org/10.1371/journal.pone.0287985 |
| spellingShingle | Shreyas Kar Lily G Bessette Richard Wyss Aaron S Kesselheim Kueiyu Joshua Lin The impact of electronic health record discontinuity on prediction modeling. PLoS ONE |
| title | The impact of electronic health record discontinuity on prediction modeling. |
| title_full | The impact of electronic health record discontinuity on prediction modeling. |
| title_fullStr | The impact of electronic health record discontinuity on prediction modeling. |
| title_full_unstemmed | The impact of electronic health record discontinuity on prediction modeling. |
| title_short | The impact of electronic health record discontinuity on prediction modeling. |
| title_sort | impact of electronic health record discontinuity on prediction modeling |
| url | https://doi.org/10.1371/journal.pone.0287985 |
| work_keys_str_mv | AT shreyaskar theimpactofelectronichealthrecorddiscontinuityonpredictionmodeling AT lilygbessette theimpactofelectronichealthrecorddiscontinuityonpredictionmodeling AT richardwyss theimpactofelectronichealthrecorddiscontinuityonpredictionmodeling AT aaronskesselheim theimpactofelectronichealthrecorddiscontinuityonpredictionmodeling AT kueiyujoshualin theimpactofelectronichealthrecorddiscontinuityonpredictionmodeling AT shreyaskar impactofelectronichealthrecorddiscontinuityonpredictionmodeling AT lilygbessette impactofelectronichealthrecorddiscontinuityonpredictionmodeling AT richardwyss impactofelectronichealthrecorddiscontinuityonpredictionmodeling AT aaronskesselheim impactofelectronichealthrecorddiscontinuityonpredictionmodeling AT kueiyujoshualin impactofelectronichealthrecorddiscontinuityonpredictionmodeling |