Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study
Objectives There are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the fi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2022-08-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/12/8/e050274.full |
| _version_ | 1811184153316556800 |
|---|---|
| author | Muhammad Faisal Donald Richardson Kevin Beatson Mohammed Mohammed Massimo Fiori |
| author_facet | Muhammad Faisal Donald Richardson Kevin Beatson Mohammed Mohammed Massimo Fiori |
| author_sort | Muhammad Faisal |
| collection | DOAJ |
| description | Objectives There are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the first electronically recorded blood test results and the National Early Warning Score (NEWS2).Design Logistic regression model development and validation study.Setting Two acute hospitals (York Hospital—model development data; Scarborough Hospital—external validation data).Participants Adult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic NEWS and blood test results recorded on admission. We used logistic regression modelling to predict the risk of in-hospital mortality using two models: (1) CARMc19_N: age+sex+NEWS2 including subcomponents+COVID19; (2) CARMc19_NB: CARMc19_N in conjunction with seven blood test results and acute kidney injury score. Model performance was evaluated according to discrimination (c-statistic), calibration (graphically) and clinical usefulness at NEWS2 thresholds of 4+, 5+, 6+.Results The risk of in-hospital mortality following emergency medical admission was similar in development and validation datasets (8.4% vs 8.2%). The c-statistics for predicting mortality for CARMc19_NB is better than CARMc19_N in the validation dataset (CARMc19_NB=0.88 (95% CI 0.86 to 0.90) vs CARMc19_N=0.86 (95% CI 0.83 to 0.88)). Both models had good calibration (CARMc19_NB=1.01 (95% CI 0.88 to 1.14) and CARMc19_N:0.95 (95% CI 0.83 to 1.06)). At all NEWS2 thresholds (4+, 5+, 6+) model, CARMc19_NB had better sensitivity and similar specificity.Conclusions We have developed a validated CARMc19 scores with good performance characteristics for predicting the risk of in-hospital mortality. Since the CARMc19 scores place no additional data collection burden on clinicians, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure. |
| first_indexed | 2024-04-11T13:08:24Z |
| format | Article |
| id | doaj.art-8ba4cfcb46a248c0bbf8aa074aaef3e7 |
| institution | Directory Open Access Journal |
| issn | 2044-6055 |
| language | English |
| last_indexed | 2024-04-11T13:08:24Z |
| publishDate | 2022-08-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj.art-8ba4cfcb46a248c0bbf8aa074aaef3e72022-12-22T04:22:39ZengBMJ Publishing GroupBMJ Open2044-60552022-08-0112810.1136/bmjopen-2021-050274Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation studyMuhammad Faisal0Donald Richardson1Kevin Beatson2Mohammed Mohammed3Massimo Fiori4Faculty of Health Studies, University of Bradford, Bradford, UKDepartment of Renal Medicine, York Teaching Hospital NHS Foundation Trust, York, UKDepartment of Information Technology, York Teaching Hospitals NHS Foundation Trust, York, UKFaculty of Health Studies, University of Bradford, Bradford, UKDepartment of Information Technology, York Teaching Hospitals NHS Foundation Trust, York, UKObjectives There are no established mortality risk equations specifically for unplanned emergency medical admissions which include patients with SARS-19 (COVID-19). We aim to develop and validate a computer-aided risk score (CARMc19) for predicting mortality risk by combining COVID-19 status, the first electronically recorded blood test results and the National Early Warning Score (NEWS2).Design Logistic regression model development and validation study.Setting Two acute hospitals (York Hospital—model development data; Scarborough Hospital—external validation data).Participants Adult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic NEWS and blood test results recorded on admission. We used logistic regression modelling to predict the risk of in-hospital mortality using two models: (1) CARMc19_N: age+sex+NEWS2 including subcomponents+COVID19; (2) CARMc19_NB: CARMc19_N in conjunction with seven blood test results and acute kidney injury score. Model performance was evaluated according to discrimination (c-statistic), calibration (graphically) and clinical usefulness at NEWS2 thresholds of 4+, 5+, 6+.Results The risk of in-hospital mortality following emergency medical admission was similar in development and validation datasets (8.4% vs 8.2%). The c-statistics for predicting mortality for CARMc19_NB is better than CARMc19_N in the validation dataset (CARMc19_NB=0.88 (95% CI 0.86 to 0.90) vs CARMc19_N=0.86 (95% CI 0.83 to 0.88)). Both models had good calibration (CARMc19_NB=1.01 (95% CI 0.88 to 1.14) and CARMc19_N:0.95 (95% CI 0.83 to 1.06)). At all NEWS2 thresholds (4+, 5+, 6+) model, CARMc19_NB had better sensitivity and similar specificity.Conclusions We have developed a validated CARMc19 scores with good performance characteristics for predicting the risk of in-hospital mortality. Since the CARMc19 scores place no additional data collection burden on clinicians, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.https://bmjopen.bmj.com/content/12/8/e050274.full |
| spellingShingle | Muhammad Faisal Donald Richardson Kevin Beatson Mohammed Mohammed Massimo Fiori Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study BMJ Open |
| title | Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study |
| title_full | Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study |
| title_fullStr | Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study |
| title_full_unstemmed | Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study |
| title_short | Development and validation of automated computer-aided risk scores to predict in-hospital mortality for emergency medical admissions with COVID-19: a retrospective cohort development and validation study |
| title_sort | development and validation of automated computer aided risk scores to predict in hospital mortality for emergency medical admissions with covid 19 a retrospective cohort development and validation study |
| url | https://bmjopen.bmj.com/content/12/8/e050274.full |
| work_keys_str_mv | AT muhammadfaisal developmentandvalidationofautomatedcomputeraidedriskscorestopredictinhospitalmortalityforemergencymedicaladmissionswithcovid19aretrospectivecohortdevelopmentandvalidationstudy AT donaldrichardson developmentandvalidationofautomatedcomputeraidedriskscorestopredictinhospitalmortalityforemergencymedicaladmissionswithcovid19aretrospectivecohortdevelopmentandvalidationstudy AT kevinbeatson developmentandvalidationofautomatedcomputeraidedriskscorestopredictinhospitalmortalityforemergencymedicaladmissionswithcovid19aretrospectivecohortdevelopmentandvalidationstudy AT mohammedmohammed developmentandvalidationofautomatedcomputeraidedriskscorestopredictinhospitalmortalityforemergencymedicaladmissionswithcovid19aretrospectivecohortdevelopmentandvalidationstudy AT massimofiori developmentandvalidationofautomatedcomputeraidedriskscorestopredictinhospitalmortalityforemergencymedicaladmissionswithcovid19aretrospectivecohortdevelopmentandvalidationstudy |