Inter hospital external validation of interpretable machine learning based triage score for the emergency department using common data model
Abstract Emergency departments (ED) are complex, triage is a main task in the ED to prioritize patient with limited medical resources who need them most. Machine learning (ML) based ED triage tool, Score for Emergency Risk Prediction (SERP), was previously developed using an interpretable ML framewo...
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-03-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-024-54364-7 |
| _version_ | 1827310189465829376 |
|---|---|
| author | Jae Yong Yu Doyeop Kim Sunyoung Yoon Taerim Kim SeJin Heo Hansol Chang Gab Soo Han Kyung Won Jeong Rae Woong Park Jun Myung Gwon Feng Xie Marcus Eng Hock Ong Yih Yng Ng Hyung Joon Joo Won Chul Cha |
| author_facet | Jae Yong Yu Doyeop Kim Sunyoung Yoon Taerim Kim SeJin Heo Hansol Chang Gab Soo Han Kyung Won Jeong Rae Woong Park Jun Myung Gwon Feng Xie Marcus Eng Hock Ong Yih Yng Ng Hyung Joon Joo Won Chul Cha |
| author_sort | Jae Yong Yu |
| collection | DOAJ |
| description | Abstract Emergency departments (ED) are complex, triage is a main task in the ED to prioritize patient with limited medical resources who need them most. Machine learning (ML) based ED triage tool, Score for Emergency Risk Prediction (SERP), was previously developed using an interpretable ML framework with single center. We aimed to develop SERP with 3 Korean multicenter cohorts based on common data model (CDM) without data sharing and compare performance with inter-hospital validation design. This retrospective cohort study included all adult emergency visit patients of 3 hospitals in Korea from 2016 to 2017. We adopted CDM for the standardized multicenter research. The outcome of interest was 2-day mortality after the patients’ ED visit. We developed each hospital SERP using interpretable ML framework and validated inter-hospital wisely. We accessed the performance of each hospital’s score based on some metrics considering data imbalance strategy. The study population for each hospital included 87,670, 83,363 and 54,423 ED visits from 2016 to 2017. The 2-day mortality rate were 0.51%, 0.56% and 0.65%. Validation results showed accurate for inter hospital validation which has at least AUROC of 0.899 (0.858–0.940). We developed multicenter based Interpretable ML model using CDM for 2-day mortality prediction and executed Inter-hospital external validation which showed enough high accuracy. |
| first_indexed | 2024-04-24T19:55:40Z |
| format | Article |
| id | doaj.art-19231b0384ed4715b51b309f4713a949 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-24T19:55:40Z |
| publishDate | 2024-03-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-19231b0384ed4715b51b309f4713a9492024-03-24T12:19:14ZengNature PortfolioScientific Reports2045-23222024-03-011411910.1038/s41598-024-54364-7Inter hospital external validation of interpretable machine learning based triage score for the emergency department using common data modelJae Yong Yu0Doyeop Kim1Sunyoung Yoon2Taerim Kim3SeJin Heo4Hansol Chang5Gab Soo Han6Kyung Won Jeong7Rae Woong Park8Jun Myung Gwon9Feng Xie10Marcus Eng Hock Ong11Yih Yng Ng12Hyung Joon Joo13Won Chul Cha14Department of Biomedical Systems Informatics, Yonsei University College of MedicineDepartment of Biomedical Informatics, Ajou University School of MedicineDepartment of Digital Health, Samsung Advanced Institute for Health Science & Technology, Sungkyunkwan UniversityDepartment of Emergency Medicine, Samsung Medical Center, Sungkyunkwan University School of MedicineDepartment of Digital Health, Samsung Advanced Institute for Health Science & Technology, Sungkyunkwan UniversityDepartment of Digital Health, Samsung Advanced Institute for Health Science & Technology, Sungkyunkwan UniversityDepartment of Cardiology, Cardiovascular Center, College of Medicine, Korea UniversityDepartment of Biomedical Informatics, Ajou University School of MedicineDepartment of Biomedical Informatics, Ajou University School of MedicineDepartment of Critical Care and Emergency Medicine, Mediplex Sejong HospitalDepartment of Biomedical Data Science, Stanford UniversityProgramme in Health Services and Systems Research, Duke–National University of Singapore Medical SchoolDigital and Smart Health Office, Tan Tock Seng HospitalDepartment of Cardiology, Cardiovascular Center, College of Medicine, Korea UniversityDepartment of Digital Health, Samsung Advanced Institute for Health Science & Technology, Sungkyunkwan UniversityAbstract Emergency departments (ED) are complex, triage is a main task in the ED to prioritize patient with limited medical resources who need them most. Machine learning (ML) based ED triage tool, Score for Emergency Risk Prediction (SERP), was previously developed using an interpretable ML framework with single center. We aimed to develop SERP with 3 Korean multicenter cohorts based on common data model (CDM) without data sharing and compare performance with inter-hospital validation design. This retrospective cohort study included all adult emergency visit patients of 3 hospitals in Korea from 2016 to 2017. We adopted CDM for the standardized multicenter research. The outcome of interest was 2-day mortality after the patients’ ED visit. We developed each hospital SERP using interpretable ML framework and validated inter-hospital wisely. We accessed the performance of each hospital’s score based on some metrics considering data imbalance strategy. The study population for each hospital included 87,670, 83,363 and 54,423 ED visits from 2016 to 2017. The 2-day mortality rate were 0.51%, 0.56% and 0.65%. Validation results showed accurate for inter hospital validation which has at least AUROC of 0.899 (0.858–0.940). We developed multicenter based Interpretable ML model using CDM for 2-day mortality prediction and executed Inter-hospital external validation which showed enough high accuracy.https://doi.org/10.1038/s41598-024-54364-7 |
| spellingShingle | Jae Yong Yu Doyeop Kim Sunyoung Yoon Taerim Kim SeJin Heo Hansol Chang Gab Soo Han Kyung Won Jeong Rae Woong Park Jun Myung Gwon Feng Xie Marcus Eng Hock Ong Yih Yng Ng Hyung Joon Joo Won Chul Cha Inter hospital external validation of interpretable machine learning based triage score for the emergency department using common data model Scientific Reports |
| title | Inter hospital external validation of interpretable machine learning based triage score for the emergency department using common data model |
| title_full | Inter hospital external validation of interpretable machine learning based triage score for the emergency department using common data model |
| title_fullStr | Inter hospital external validation of interpretable machine learning based triage score for the emergency department using common data model |
| title_full_unstemmed | Inter hospital external validation of interpretable machine learning based triage score for the emergency department using common data model |
| title_short | Inter hospital external validation of interpretable machine learning based triage score for the emergency department using common data model |
| title_sort | inter hospital external validation of interpretable machine learning based triage score for the emergency department using common data model |
| url | https://doi.org/10.1038/s41598-024-54364-7 |
| work_keys_str_mv | AT jaeyongyu interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT doyeopkim interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT sunyoungyoon interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT taerimkim interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT sejinheo interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT hansolchang interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT gabsoohan interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT kyungwonjeong interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT raewoongpark interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT junmyunggwon interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT fengxie interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT marcusenghockong interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT yihyngng interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT hyungjoonjoo interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel AT wonchulcha interhospitalexternalvalidationofinterpretablemachinelearningbasedtriagescorefortheemergencydepartmentusingcommondatamodel |