Using Machine Learning to Develop and Validate an In-Hospital Mortality Prediction Model for Patients with Suspected Sepsis
Background: Early recognition of sepsis and the prediction of mortality in patients with infection are important. This multi-center, ED-based study aimed to develop and validate a 28-day mortality prediction model for patients with infection using various machine learning (ML) algorithms. Methods: P...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Biomedicines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-9059/10/4/802 |
| _version_ | 1797436841787392000 |
|---|---|
| author | Hsiao-Yun Chao Chin-Chieh Wu Avichandra Singh Andrew Shedd Jon Wolfshohl Eric H. Chou Yhu-Chering Huang Kuan-Fu Chen |
| author_facet | Hsiao-Yun Chao Chin-Chieh Wu Avichandra Singh Andrew Shedd Jon Wolfshohl Eric H. Chou Yhu-Chering Huang Kuan-Fu Chen |
| author_sort | Hsiao-Yun Chao |
| collection | DOAJ |
| description | Background: Early recognition of sepsis and the prediction of mortality in patients with infection are important. This multi-center, ED-based study aimed to develop and validate a 28-day mortality prediction model for patients with infection using various machine learning (ML) algorithms. Methods: Patients with acute infection requiring intravenous antibiotic treatment during the first 24 h of admission were prospectively recruited. Patient demographics, comorbidities, clinical signs and symptoms, laboratory test data, selected sepsis-related novel biomarkers, and 28-day mortality were collected and divided into training (70%) and testing (30%) datasets. Logistic regression and seven ML algorithms were used to develop the prediction models. The area under the receiver operating characteristic curve (AUROC) was used to compare different models. Results: A total of 555 patients were recruited with a full panel of biomarker tests. Among them, 18% fulfilled Sepsis-3 criteria, with a 28-day mortality rate of 8%. The wrapper algorithm selected 30 features, including disease severity scores, biochemical parameters, and conventional and few sepsis-related biomarkers. Random forest outperformed other ML models (AUROC: 0.96; 95% confidence interval: 0.93–0.98) and SOFA and early warning scores (AUROC: 0.64–0.84) in the prediction of 28-day mortality in patients with infection. Additionally, random forest remained the best-performing model, with an AUROC of 0.95 (95% CI: 0.91–0.98, <i>p</i> = 0.725) after removing five sepsis-related novel biomarkers. Conclusions: Our results demonstrated that ML models provide a more accurate prediction of 28-day mortality with an enhanced ability in dealing with multi-dimensional data than the logistic regression model. |
| first_indexed | 2024-03-09T11:07:27Z |
| format | Article |
| id | doaj.art-baf2524c6e2b4c2a9a23f74db30d37d2 |
| institution | Directory Open Access Journal |
| issn | 2227-9059 |
| language | English |
| last_indexed | 2024-03-09T11:07:27Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Biomedicines |
| spelling | doaj.art-baf2524c6e2b4c2a9a23f74db30d37d22023-12-01T00:53:33ZengMDPI AGBiomedicines2227-90592022-03-0110480210.3390/biomedicines10040802Using Machine Learning to Develop and Validate an In-Hospital Mortality Prediction Model for Patients with Suspected SepsisHsiao-Yun Chao0Chin-Chieh Wu1Avichandra Singh2Andrew Shedd3Jon Wolfshohl4Eric H. Chou5Yhu-Chering Huang6Kuan-Fu Chen7Department of Emergency Medicine, Linkou Chang Gung Memorial Hospital, No. 5, Fu-Shin Street, Gueishan Village, Taoyuan 333423, TaiwanClinical Informatics and Medical Statistics Research Center, Chang Gung University, Taoyuan 33302, TaiwanDepartment of Emergency Medicine, Keelung Chang Gung Memorial Hospital, Keelung 20401, TaiwanDepartment of Emergency Medicine, Baylor Scott and White All Saints Medical Center, Fort Worth, TX 76104, USADepartment of Emergency Medicine, Baylor Scott and White All Saints Medical Center, Fort Worth, TX 76104, USADepartment of Emergency Medicine, Baylor Scott and White All Saints Medical Center, Fort Worth, TX 76104, USADivision of Pediatric Infectious Diseases, Linkou Chang Gung Memorial Hospital, No. 5, Fu-Shin Street, Gueishan Village, Taoyuan 333423, TaiwanDepartment of Emergency Medicine, Linkou Chang Gung Memorial Hospital, No. 5, Fu-Shin Street, Gueishan Village, Taoyuan 333423, TaiwanBackground: Early recognition of sepsis and the prediction of mortality in patients with infection are important. This multi-center, ED-based study aimed to develop and validate a 28-day mortality prediction model for patients with infection using various machine learning (ML) algorithms. Methods: Patients with acute infection requiring intravenous antibiotic treatment during the first 24 h of admission were prospectively recruited. Patient demographics, comorbidities, clinical signs and symptoms, laboratory test data, selected sepsis-related novel biomarkers, and 28-day mortality were collected and divided into training (70%) and testing (30%) datasets. Logistic regression and seven ML algorithms were used to develop the prediction models. The area under the receiver operating characteristic curve (AUROC) was used to compare different models. Results: A total of 555 patients were recruited with a full panel of biomarker tests. Among them, 18% fulfilled Sepsis-3 criteria, with a 28-day mortality rate of 8%. The wrapper algorithm selected 30 features, including disease severity scores, biochemical parameters, and conventional and few sepsis-related biomarkers. Random forest outperformed other ML models (AUROC: 0.96; 95% confidence interval: 0.93–0.98) and SOFA and early warning scores (AUROC: 0.64–0.84) in the prediction of 28-day mortality in patients with infection. Additionally, random forest remained the best-performing model, with an AUROC of 0.95 (95% CI: 0.91–0.98, <i>p</i> = 0.725) after removing five sepsis-related novel biomarkers. Conclusions: Our results demonstrated that ML models provide a more accurate prediction of 28-day mortality with an enhanced ability in dealing with multi-dimensional data than the logistic regression model.https://www.mdpi.com/2227-9059/10/4/802biomarkerlogistic regressionmachine learningmortality predictionsepsis |
| spellingShingle | Hsiao-Yun Chao Chin-Chieh Wu Avichandra Singh Andrew Shedd Jon Wolfshohl Eric H. Chou Yhu-Chering Huang Kuan-Fu Chen Using Machine Learning to Develop and Validate an In-Hospital Mortality Prediction Model for Patients with Suspected Sepsis Biomedicines biomarker logistic regression machine learning mortality prediction sepsis |
| title | Using Machine Learning to Develop and Validate an In-Hospital Mortality Prediction Model for Patients with Suspected Sepsis |
| title_full | Using Machine Learning to Develop and Validate an In-Hospital Mortality Prediction Model for Patients with Suspected Sepsis |
| title_fullStr | Using Machine Learning to Develop and Validate an In-Hospital Mortality Prediction Model for Patients with Suspected Sepsis |
| title_full_unstemmed | Using Machine Learning to Develop and Validate an In-Hospital Mortality Prediction Model for Patients with Suspected Sepsis |
| title_short | Using Machine Learning to Develop and Validate an In-Hospital Mortality Prediction Model for Patients with Suspected Sepsis |
| title_sort | using machine learning to develop and validate an in hospital mortality prediction model for patients with suspected sepsis |
| topic | biomarker logistic regression machine learning mortality prediction sepsis |
| url | https://www.mdpi.com/2227-9059/10/4/802 |
| work_keys_str_mv | AT hsiaoyunchao usingmachinelearningtodevelopandvalidateaninhospitalmortalitypredictionmodelforpatientswithsuspectedsepsis AT chinchiehwu usingmachinelearningtodevelopandvalidateaninhospitalmortalitypredictionmodelforpatientswithsuspectedsepsis AT avichandrasingh usingmachinelearningtodevelopandvalidateaninhospitalmortalitypredictionmodelforpatientswithsuspectedsepsis AT andrewshedd usingmachinelearningtodevelopandvalidateaninhospitalmortalitypredictionmodelforpatientswithsuspectedsepsis AT jonwolfshohl usingmachinelearningtodevelopandvalidateaninhospitalmortalitypredictionmodelforpatientswithsuspectedsepsis AT erichchou usingmachinelearningtodevelopandvalidateaninhospitalmortalitypredictionmodelforpatientswithsuspectedsepsis AT yhucheringhuang usingmachinelearningtodevelopandvalidateaninhospitalmortalitypredictionmodelforpatientswithsuspectedsepsis AT kuanfuchen usingmachinelearningtodevelopandvalidateaninhospitalmortalitypredictionmodelforpatientswithsuspectedsepsis |