Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Emergency department triage is the first point in time when a pa...
| Main Authors: | , , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2020
|
| Online Access: | https://hdl.handle.net/1721.1/125378 |
| _version_ | 1826216973021741056 |
|---|---|
| author | Fernandes, Marta Mendes, Rúben Vieira, Susana M. Leite, Francisca Palos, Carlos Johnson, Alistair Edward William Finkelstein, Stan Neil Horng, Steven Celi, Leo Anthony |
| author2 | Harvard University--MIT Division of Health Sciences and Technology |
| author_facet | Harvard University--MIT Division of Health Sciences and Technology Fernandes, Marta Mendes, Rúben Vieira, Susana M. Leite, Francisca Palos, Carlos Johnson, Alistair Edward William Finkelstein, Stan Neil Horng, Steven Celi, Leo Anthony |
| author_sort | Fernandes, Marta |
| collection | MIT |
| description | This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Emergency department triage is the first point in time when a patient’s acuity level is determined. The time to assign a priority at triage is short and it is vital to accurately stratify patients at this stage, since under-triage can lead to increased morbidity, mortality and costs. Our aim was to present a model that can assist healthcare professionals in triage decision making, namely in the stratification of patients through the risk prediction of a composite critical outcome—mortality and cardiopulmonary arrest. Our study cohort consisted of 235826 adult patients triaged at a Portuguese Emergency Department from 2012 to 2016. Patients were assigned to emergent, very urgent or urgent priorities of the Manchester Triage System (MTS). Demographics, clinical variables routinely collected at triage and the patients’ chief complaint were used. Logistic regression, random forests and extreme gradient boosting were developed using all available variables. The term frequency–inverse document frequency (TF-IDF) natural language processing weighting factor was applied to vectorize the chief complaint. Stratified random sampling was used to split the data into train (70%) and test (30%) data sets. Ten-fold cross validation was performed in train to optimize model hyper-parameters. The performance obtained with the best model was compared against the reference model—a regularized logistic regression trained using only triage priorities. Extreme gradient boosting exhibited good calibration properties and yielded areas under the receiver operating characteristic and precision-recall curves of 0.96 (95% CI 0.95-0.97) and 0.31 (95% CI 0.26-0.36), respectively. The predictors ranked with higher importance by this model were the Glasgow coma score, the patients’ age, pulse oximetry and arrival mode. Compared to the reference, the extreme gradient boosting model using clinical variables and the chief complaint presented higher recall for patients assigned MTS-3 and can identify those who are at risk of the composite outcome. |
| first_indexed | 2024-09-23T16:56:07Z |
| format | Article |
| id | mit-1721.1/125378 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T16:56:07Z |
| publishDate | 2020 |
| publisher | Public Library of Science (PLoS) |
| record_format | dspace |
| spelling | mit-1721.1/1253782022-10-03T09:15:43Z Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing Fernandes, Marta Mendes, Rúben Vieira, Susana M. Leite, Francisca Palos, Carlos Johnson, Alistair Edward William Finkelstein, Stan Neil Horng, Steven Celi, Leo Anthony Harvard University--MIT Division of Health Sciences and Technology Massachusetts Institute of Technology. Institute for Data, Systems, and Society Massachusetts Institute of Technology. Institute for Medical Engineering & Science This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Emergency department triage is the first point in time when a patient’s acuity level is determined. The time to assign a priority at triage is short and it is vital to accurately stratify patients at this stage, since under-triage can lead to increased morbidity, mortality and costs. Our aim was to present a model that can assist healthcare professionals in triage decision making, namely in the stratification of patients through the risk prediction of a composite critical outcome—mortality and cardiopulmonary arrest. Our study cohort consisted of 235826 adult patients triaged at a Portuguese Emergency Department from 2012 to 2016. Patients were assigned to emergent, very urgent or urgent priorities of the Manchester Triage System (MTS). Demographics, clinical variables routinely collected at triage and the patients’ chief complaint were used. Logistic regression, random forests and extreme gradient boosting were developed using all available variables. The term frequency–inverse document frequency (TF-IDF) natural language processing weighting factor was applied to vectorize the chief complaint. Stratified random sampling was used to split the data into train (70%) and test (30%) data sets. Ten-fold cross validation was performed in train to optimize model hyper-parameters. The performance obtained with the best model was compared against the reference model—a regularized logistic regression trained using only triage priorities. Extreme gradient boosting exhibited good calibration properties and yielded areas under the receiver operating characteristic and precision-recall curves of 0.96 (95% CI 0.95-0.97) and 0.31 (95% CI 0.26-0.36), respectively. The predictors ranked with higher importance by this model were the Glasgow coma score, the patients’ age, pulse oximetry and arrival mode. Compared to the reference, the extreme gradient boosting model using clinical variables and the chief complaint presented higher recall for patients assigned MTS-3 and can identify those who are at risk of the composite outcome. 2020-05-21T15:37:13Z 2020-05-21T15:37:13Z 2020-04 2019-10 2020-05-14T17:28:50Z Article http://purl.org/eprint/type/JournalArticle 1932-6203 https://hdl.handle.net/1721.1/125378 Fernandes M. et al. "Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing." PLoS ONE 15, 4 (April 2020): e0230876 © 2020 Fernandes et al. en http://dx.doi.org/10.1371/journal.pone.0230876 PLoS ONE Creative Commons Attribution 4.0 International license https://creativecommons.org/licenses/by/4.0/ application/pdf Public Library of Science (PLoS) PLoS |
| spellingShingle | Fernandes, Marta Mendes, Rúben Vieira, Susana M. Leite, Francisca Palos, Carlos Johnson, Alistair Edward William Finkelstein, Stan Neil Horng, Steven Celi, Leo Anthony Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing |
| title | Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing |
| title_full | Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing |
| title_fullStr | Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing |
| title_full_unstemmed | Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing |
| title_short | Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing |
| title_sort | risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing |
| url | https://hdl.handle.net/1721.1/125378 |
| work_keys_str_mv | AT fernandesmarta riskofmortalityandcardiopulmonaryarrestincriticalpatientspresentingtotheemergencydepartmentusingmachinelearningandnaturallanguageprocessing AT mendesruben riskofmortalityandcardiopulmonaryarrestincriticalpatientspresentingtotheemergencydepartmentusingmachinelearningandnaturallanguageprocessing AT vieirasusanam riskofmortalityandcardiopulmonaryarrestincriticalpatientspresentingtotheemergencydepartmentusingmachinelearningandnaturallanguageprocessing AT leitefrancisca riskofmortalityandcardiopulmonaryarrestincriticalpatientspresentingtotheemergencydepartmentusingmachinelearningandnaturallanguageprocessing AT paloscarlos riskofmortalityandcardiopulmonaryarrestincriticalpatientspresentingtotheemergencydepartmentusingmachinelearningandnaturallanguageprocessing AT johnsonalistairedwardwilliam riskofmortalityandcardiopulmonaryarrestincriticalpatientspresentingtotheemergencydepartmentusingmachinelearningandnaturallanguageprocessing AT finkelsteinstanneil riskofmortalityandcardiopulmonaryarrestincriticalpatientspresentingtotheemergencydepartmentusingmachinelearningandnaturallanguageprocessing AT horngsteven riskofmortalityandcardiopulmonaryarrestincriticalpatientspresentingtotheemergencydepartmentusingmachinelearningandnaturallanguageprocessing AT celileoanthony riskofmortalityandcardiopulmonaryarrestincriticalpatientspresentingtotheemergencydepartmentusingmachinelearningandnaturallanguageprocessing |