Machine learning-based model for prediction of clinical deterioration in hospitalized patients by COVID 19
Abstract Despite the publication of great number of tools to aid decisions in COVID-19 patients, there is a lack of good instruments to predict clinical deterioration. COVID19-Osakidetza is a prospective cohort study recruiting COVID-19 patients. We collected information from baseline to discharge o...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-05-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-09771-z |
| _version_ | 1818250750580490240 |
|---|---|
| author | Susana Garcia-Gutiérrez Cristobal Esteban-Aizpiri Iratxe Lafuente Irantzu Barrio Raul Quiros Jose Maria Quintana Ane Uranga COVID-REDISSEC Working Group |
| author_facet | Susana Garcia-Gutiérrez Cristobal Esteban-Aizpiri Iratxe Lafuente Irantzu Barrio Raul Quiros Jose Maria Quintana Ane Uranga COVID-REDISSEC Working Group |
| author_sort | Susana Garcia-Gutiérrez |
| collection | DOAJ |
| description | Abstract Despite the publication of great number of tools to aid decisions in COVID-19 patients, there is a lack of good instruments to predict clinical deterioration. COVID19-Osakidetza is a prospective cohort study recruiting COVID-19 patients. We collected information from baseline to discharge on: sociodemographic characteristics, comorbidities and associated medications, vital signs, treatment received and lab test results. Outcome was need for intensive ventilatory support (with at least standard high-flow oxygen face mask with a reservoir bag for at least 6 h and need for more intensive therapy afterwards or Optiflow high-flow nasal cannula or noninvasive or invasive mechanical ventilation) and/or admission to a critical care unit and/or death during hospitalization. We developed a Catboost model summarizing the findings using Shapley Additive Explanations. Performance of the model was assessed using area under the receiver operating characteristic and prediction recall curves (AUROC and AUPRC respectively) and calibrated using the Hosmer–Lemeshow test. Overall, 1568 patients were included in the derivation cohort and 956 in the (external) validation cohort. The percentages of patients who reached the composite endpoint were 23.3% vs 20% respectively. The strongest predictors of clinical deterioration were arterial blood oxygen pressure, followed by age, levels of several markers of inflammation (procalcitonin, LDH, CRP) and alterations in blood count and coagulation. Some medications, namely, ATC AO2 (antiacids) and N05 (neuroleptics) were also among the group of main predictors, together with C03 (diuretics). In the validation set, the CatBoost AUROC was 0.79, AUPRC 0.21 and Hosmer–Lemeshow test statistic 0.36. We present a machine learning-based prediction model with excellent performance properties to implement in EHRs. Our main goal was to predict progression to a score of 5 or higher on the WHO Clinical Progression Scale before patients required mechanical ventilation. Future steps are to externally validate the model in other settings and in a cohort from a different period and to apply the algorithm in clinical practice. Registration: ClinicalTrials.gov Identifier: NCT04463706. |
| first_indexed | 2024-12-12T15:57:21Z |
| format | Article |
| id | doaj.art-b62cc0ab553c497abbabc6d1c4622315 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-12-12T15:57:21Z |
| publishDate | 2022-05-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-b62cc0ab553c497abbabc6d1c46223152022-12-22T00:19:28ZengNature PortfolioScientific Reports2045-23222022-05-0112111110.1038/s41598-022-09771-zMachine learning-based model for prediction of clinical deterioration in hospitalized patients by COVID 19Susana Garcia-Gutiérrez0Cristobal Esteban-Aizpiri1Iratxe Lafuente2Irantzu Barrio3Raul Quiros4Jose Maria Quintana5Ane Uranga6COVID-REDISSEC Working GroupOsakidetza Basque Health Service, Research Unit, Galdakao-Usansolo University HospitalCambrian IntelligenceOsakidetza Basque Health Service, Research Unit, Galdakao-Usansolo University HospitalDepartment of Applied Mathematics and Operational Research, University of the Basque CountryInternal Medicine Service, Hospital Costa del SolOsakidetza Basque Health Service, Research Unit, Galdakao-Usansolo University HospitalOsakidetza Basque Health Service, Respiratory Service, Galdakao-Usansolo University HospitalAbstract Despite the publication of great number of tools to aid decisions in COVID-19 patients, there is a lack of good instruments to predict clinical deterioration. COVID19-Osakidetza is a prospective cohort study recruiting COVID-19 patients. We collected information from baseline to discharge on: sociodemographic characteristics, comorbidities and associated medications, vital signs, treatment received and lab test results. Outcome was need for intensive ventilatory support (with at least standard high-flow oxygen face mask with a reservoir bag for at least 6 h and need for more intensive therapy afterwards or Optiflow high-flow nasal cannula or noninvasive or invasive mechanical ventilation) and/or admission to a critical care unit and/or death during hospitalization. We developed a Catboost model summarizing the findings using Shapley Additive Explanations. Performance of the model was assessed using area under the receiver operating characteristic and prediction recall curves (AUROC and AUPRC respectively) and calibrated using the Hosmer–Lemeshow test. Overall, 1568 patients were included in the derivation cohort and 956 in the (external) validation cohort. The percentages of patients who reached the composite endpoint were 23.3% vs 20% respectively. The strongest predictors of clinical deterioration were arterial blood oxygen pressure, followed by age, levels of several markers of inflammation (procalcitonin, LDH, CRP) and alterations in blood count and coagulation. Some medications, namely, ATC AO2 (antiacids) and N05 (neuroleptics) were also among the group of main predictors, together with C03 (diuretics). In the validation set, the CatBoost AUROC was 0.79, AUPRC 0.21 and Hosmer–Lemeshow test statistic 0.36. We present a machine learning-based prediction model with excellent performance properties to implement in EHRs. Our main goal was to predict progression to a score of 5 or higher on the WHO Clinical Progression Scale before patients required mechanical ventilation. Future steps are to externally validate the model in other settings and in a cohort from a different period and to apply the algorithm in clinical practice. Registration: ClinicalTrials.gov Identifier: NCT04463706.https://doi.org/10.1038/s41598-022-09771-z |
| spellingShingle | Susana Garcia-Gutiérrez Cristobal Esteban-Aizpiri Iratxe Lafuente Irantzu Barrio Raul Quiros Jose Maria Quintana Ane Uranga COVID-REDISSEC Working Group Machine learning-based model for prediction of clinical deterioration in hospitalized patients by COVID 19 Scientific Reports |
| title | Machine learning-based model for prediction of clinical deterioration in hospitalized patients by COVID 19 |
| title_full | Machine learning-based model for prediction of clinical deterioration in hospitalized patients by COVID 19 |
| title_fullStr | Machine learning-based model for prediction of clinical deterioration in hospitalized patients by COVID 19 |
| title_full_unstemmed | Machine learning-based model for prediction of clinical deterioration in hospitalized patients by COVID 19 |
| title_short | Machine learning-based model for prediction of clinical deterioration in hospitalized patients by COVID 19 |
| title_sort | machine learning based model for prediction of clinical deterioration in hospitalized patients by covid 19 |
| url | https://doi.org/10.1038/s41598-022-09771-z |
| work_keys_str_mv | AT susanagarciagutierrez machinelearningbasedmodelforpredictionofclinicaldeteriorationinhospitalizedpatientsbycovid19 AT cristobalestebanaizpiri machinelearningbasedmodelforpredictionofclinicaldeteriorationinhospitalizedpatientsbycovid19 AT iratxelafuente machinelearningbasedmodelforpredictionofclinicaldeteriorationinhospitalizedpatientsbycovid19 AT irantzubarrio machinelearningbasedmodelforpredictionofclinicaldeteriorationinhospitalizedpatientsbycovid19 AT raulquiros machinelearningbasedmodelforpredictionofclinicaldeteriorationinhospitalizedpatientsbycovid19 AT josemariaquintana machinelearningbasedmodelforpredictionofclinicaldeteriorationinhospitalizedpatientsbycovid19 AT aneuranga machinelearningbasedmodelforpredictionofclinicaldeteriorationinhospitalizedpatientsbycovid19 AT covidredissecworkinggroup machinelearningbasedmodelforpredictionofclinicaldeteriorationinhospitalizedpatientsbycovid19 |