Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID‐19—A retrospective, multicentre study
Abstract Introduction COVID‐19 virus has undergone mutations, and the introduction of vaccines and effective treatments have changed its clinical severity. We hypothesized that models that evolve may better predict invasive mechanical ventilation or death than do static models. Methods This retrospe...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-01-01
|
| Series: | The Clinical Respiratory Journal |
| Subjects: | |
| Online Access: | https://doi.org/10.1111/crj.13560 |
| _version_ | 1797957243798290432 |
|---|---|
| author | Milo Engoren Carlo Pancaro Nicholas S. Yeldo Lotfi S. Kerzabi Nicholas Douville |
| author_facet | Milo Engoren Carlo Pancaro Nicholas S. Yeldo Lotfi S. Kerzabi Nicholas Douville |
| author_sort | Milo Engoren |
| collection | DOAJ |
| description | Abstract Introduction COVID‐19 virus has undergone mutations, and the introduction of vaccines and effective treatments have changed its clinical severity. We hypothesized that models that evolve may better predict invasive mechanical ventilation or death than do static models. Methods This retrospective study of adult patients with COVID‐19 from six Michigan hospitals analysed 20 demographic, comorbid, vital sign and laboratory factors, one derived factor and nine factors representing changes in vital signs or laboratory values with time for their ability to predict death or invasive mechanical ventilation within the next 4, 8 or 24 h. Static logistic regression was constructed on the initial 300 patients and tested on the remaining 6741 patients. Rolling logistic regression was similarly constructed on the initial 300 patients, but then new patients were added, and older patients removed. Each new construction model was subsequently tested on the next patient. Static and rolling models were compared with receiver operator characteristic and precision‐recall curves. Results Of the 7041 patients, 534 (7.6%) required invasive mechanical ventilation or died within 14 days of arrival. Rolling models improved discrimination (0.865 ± 0.010, 0.856 ± 0.007 and 0.843 ± 0.005 for the 4, 8 and 24‐h models, respectively; all p < 0.001 compared with the static logistic regressions with 0.827 ± 0.011, 0.794 ± 0.012 and 0.735 ± 0.012, respectively). Similarly, the areas under the precision‐recall curves improved from 0.006, 0.010 and 0.021 with the static models to 0.030, 0.045 and 0.076 for the 4‐, 8‐ and 24‐h rolling models, respectively, all p < 0.001. Conclusion Rolling models with contemporaneous data maintained better metrics of performance than static models, which used older data. |
| first_indexed | 2024-04-11T00:01:18Z |
| format | Article |
| id | doaj.art-95f3f59ac20c41a793f99e07e21bd918 |
| institution | Directory Open Access Journal |
| issn | 1752-6981 1752-699X |
| language | English |
| last_indexed | 2024-04-11T00:01:18Z |
| publishDate | 2023-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | The Clinical Respiratory Journal |
| spelling | doaj.art-95f3f59ac20c41a793f99e07e21bd9182023-01-10T03:15:15ZengWileyThe Clinical Respiratory Journal1752-69811752-699X2023-01-01171404910.1111/crj.13560Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID‐19—A retrospective, multicentre studyMilo Engoren0Carlo Pancaro1Nicholas S. Yeldo2Lotfi S. Kerzabi3Nicholas Douville4Department of Anesthesiology University of Michigan Ann Arbor Michigan USADepartment of Anesthesiology University of Michigan Ann Arbor Michigan USADepartment of Anesthesiology Henry Ford Medical Center Detroit Michigan USADepartment of Anesthesiology Henry Ford Medical Center Detroit Michigan USADepartment of Anesthesiology University of Michigan Ann Arbor Michigan USAAbstract Introduction COVID‐19 virus has undergone mutations, and the introduction of vaccines and effective treatments have changed its clinical severity. We hypothesized that models that evolve may better predict invasive mechanical ventilation or death than do static models. Methods This retrospective study of adult patients with COVID‐19 from six Michigan hospitals analysed 20 demographic, comorbid, vital sign and laboratory factors, one derived factor and nine factors representing changes in vital signs or laboratory values with time for their ability to predict death or invasive mechanical ventilation within the next 4, 8 or 24 h. Static logistic regression was constructed on the initial 300 patients and tested on the remaining 6741 patients. Rolling logistic regression was similarly constructed on the initial 300 patients, but then new patients were added, and older patients removed. Each new construction model was subsequently tested on the next patient. Static and rolling models were compared with receiver operator characteristic and precision‐recall curves. Results Of the 7041 patients, 534 (7.6%) required invasive mechanical ventilation or died within 14 days of arrival. Rolling models improved discrimination (0.865 ± 0.010, 0.856 ± 0.007 and 0.843 ± 0.005 for the 4, 8 and 24‐h models, respectively; all p < 0.001 compared with the static logistic regressions with 0.827 ± 0.011, 0.794 ± 0.012 and 0.735 ± 0.012, respectively). Similarly, the areas under the precision‐recall curves improved from 0.006, 0.010 and 0.021 with the static models to 0.030, 0.045 and 0.076 for the 4‐, 8‐ and 24‐h rolling models, respectively, all p < 0.001. Conclusion Rolling models with contemporaneous data maintained better metrics of performance than static models, which used older data.https://doi.org/10.1111/crj.13560COVIDdeath, clinical decision support, clinical prediction modelslogistic regressionmechanical ventilation |
| spellingShingle | Milo Engoren Carlo Pancaro Nicholas S. Yeldo Lotfi S. Kerzabi Nicholas Douville Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID‐19—A retrospective, multicentre study The Clinical Respiratory Journal COVID death, clinical decision support, clinical prediction models logistic regression mechanical ventilation |
| title | Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID‐19—A retrospective, multicentre study |
| title_full | Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID‐19—A retrospective, multicentre study |
| title_fullStr | Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID‐19—A retrospective, multicentre study |
| title_full_unstemmed | Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID‐19—A retrospective, multicentre study |
| title_short | Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID‐19—A retrospective, multicentre study |
| title_sort | comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from covid 19 a retrospective multicentre study |
| topic | COVID death, clinical decision support, clinical prediction models logistic regression mechanical ventilation |
| url | https://doi.org/10.1111/crj.13560 |
| work_keys_str_mv | AT miloengoren comparisonofstaticandrollinglogisticregressionmodelsonpredictinginvasivemechanicalventilationordeathfromcovid19aretrospectivemulticentrestudy AT carlopancaro comparisonofstaticandrollinglogisticregressionmodelsonpredictinginvasivemechanicalventilationordeathfromcovid19aretrospectivemulticentrestudy AT nicholassyeldo comparisonofstaticandrollinglogisticregressionmodelsonpredictinginvasivemechanicalventilationordeathfromcovid19aretrospectivemulticentrestudy AT lotfiskerzabi comparisonofstaticandrollinglogisticregressionmodelsonpredictinginvasivemechanicalventilationordeathfromcovid19aretrospectivemulticentrestudy AT nicholasdouville comparisonofstaticandrollinglogisticregressionmodelsonpredictinginvasivemechanicalventilationordeathfromcovid19aretrospectivemulticentrestudy |