Development and validation of machine learning models for venous thromboembolism risk assessment at admission: a retrospective study
IntroductionVenous thromboembolism (VTE) risk assessment at admission is of great importance for early screening and timely prophylaxis and management during hospitalization. The purpose of this study is to develop and validate novel risk assessment models at admission based on machine learning (ML)...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-08-01
|
| Series: | Frontiers in Cardiovascular Medicine |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fcvm.2023.1198526/full |
| _version_ | 1797733123254910976 |
|---|---|
| author | Wenbo Sheng Xiaoli Wang Wenxiang Xu Zedong Hao Handong Ma Shaodian Zhang Shaodian Zhang |
| author_facet | Wenbo Sheng Xiaoli Wang Wenxiang Xu Zedong Hao Handong Ma Shaodian Zhang Shaodian Zhang |
| author_sort | Wenbo Sheng |
| collection | DOAJ |
| description | IntroductionVenous thromboembolism (VTE) risk assessment at admission is of great importance for early screening and timely prophylaxis and management during hospitalization. The purpose of this study is to develop and validate novel risk assessment models at admission based on machine learning (ML) methods.MethodsIn this retrospective study, a total of 3078 individuals were included with their Caprini variables within 24 hours at admission. Then several ML models were built, including logistic regression (LR), random forest (RF), and extreme gradient boosting (XGB). The prediction performance of ML models and the Caprini risk score (CRS) was then validated and compared through a series of evaluation metrics.ResultsThe values of AUROC and AUPRC were 0.798 and 0.303 for LR, 0.804 and 0.360 for RF, and 0.796 and 0.352 for XGB, respectively, which outperformed CRS significantly (0.714 and 0.180, P < 0.001). When prediction scores were stratified into three risk levels for application, RF could obtain more reasonable results than CRS, including smaller false positive alerts and larger lower-risk proportions. The boosting results of stratification were further verified by the net-reclassification-improvement (NRI) analysis.DiscussionThis study indicated that machine learning models could improve VTE risk prediction at admission compared with CRS. Among the ML models, RF was found to have superior performance and great potential in clinical practice. |
| first_indexed | 2024-03-12T12:24:35Z |
| format | Article |
| id | doaj.art-503be12342e64e0b9f726fc82bc4ec25 |
| institution | Directory Open Access Journal |
| issn | 2297-055X |
| language | English |
| last_indexed | 2024-03-12T12:24:35Z |
| publishDate | 2023-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Cardiovascular Medicine |
| spelling | doaj.art-503be12342e64e0b9f726fc82bc4ec252023-08-30T04:09:34ZengFrontiers Media S.A.Frontiers in Cardiovascular Medicine2297-055X2023-08-011010.3389/fcvm.2023.11985261198526Development and validation of machine learning models for venous thromboembolism risk assessment at admission: a retrospective studyWenbo Sheng0Xiaoli Wang1Wenxiang Xu2Zedong Hao3Handong Ma4Shaodian Zhang5Shaodian Zhang6Research and Development Department, Shanghai Synyi Medical Technology Co., Ltd., Shanghai, ChinaPudong Institute for Health Development, Shanghai, ChinaResearch and Development Department, Shanghai Synyi Medical Technology Co., Ltd., Shanghai, ChinaResearch and Development Department, Shanghai Synyi Medical Technology Co., Ltd., Shanghai, ChinaResearch and Development Department, Shanghai Synyi Medical Technology Co., Ltd., Shanghai, ChinaResearch and Development Department, Shanghai Synyi Medical Technology Co., Ltd., Shanghai, ChinaDivision of Medical Affairs, Shanghai Tenth People's Hospital, Shanghai, ChinaIntroductionVenous thromboembolism (VTE) risk assessment at admission is of great importance for early screening and timely prophylaxis and management during hospitalization. The purpose of this study is to develop and validate novel risk assessment models at admission based on machine learning (ML) methods.MethodsIn this retrospective study, a total of 3078 individuals were included with their Caprini variables within 24 hours at admission. Then several ML models were built, including logistic regression (LR), random forest (RF), and extreme gradient boosting (XGB). The prediction performance of ML models and the Caprini risk score (CRS) was then validated and compared through a series of evaluation metrics.ResultsThe values of AUROC and AUPRC were 0.798 and 0.303 for LR, 0.804 and 0.360 for RF, and 0.796 and 0.352 for XGB, respectively, which outperformed CRS significantly (0.714 and 0.180, P < 0.001). When prediction scores were stratified into three risk levels for application, RF could obtain more reasonable results than CRS, including smaller false positive alerts and larger lower-risk proportions. The boosting results of stratification were further verified by the net-reclassification-improvement (NRI) analysis.DiscussionThis study indicated that machine learning models could improve VTE risk prediction at admission compared with CRS. Among the ML models, RF was found to have superior performance and great potential in clinical practice.https://www.frontiersin.org/articles/10.3389/fcvm.2023.1198526/fullvenous thromboembolismrisk assessment modelmachine learningpredictive modelingrisk stratification |
| spellingShingle | Wenbo Sheng Xiaoli Wang Wenxiang Xu Zedong Hao Handong Ma Shaodian Zhang Shaodian Zhang Development and validation of machine learning models for venous thromboembolism risk assessment at admission: a retrospective study Frontiers in Cardiovascular Medicine venous thromboembolism risk assessment model machine learning predictive modeling risk stratification |
| title | Development and validation of machine learning models for venous thromboembolism risk assessment at admission: a retrospective study |
| title_full | Development and validation of machine learning models for venous thromboembolism risk assessment at admission: a retrospective study |
| title_fullStr | Development and validation of machine learning models for venous thromboembolism risk assessment at admission: a retrospective study |
| title_full_unstemmed | Development and validation of machine learning models for venous thromboembolism risk assessment at admission: a retrospective study |
| title_short | Development and validation of machine learning models for venous thromboembolism risk assessment at admission: a retrospective study |
| title_sort | development and validation of machine learning models for venous thromboembolism risk assessment at admission a retrospective study |
| topic | venous thromboembolism risk assessment model machine learning predictive modeling risk stratification |
| url | https://www.frontiersin.org/articles/10.3389/fcvm.2023.1198526/full |
| work_keys_str_mv | AT wenbosheng developmentandvalidationofmachinelearningmodelsforvenousthromboembolismriskassessmentatadmissionaretrospectivestudy AT xiaoliwang developmentandvalidationofmachinelearningmodelsforvenousthromboembolismriskassessmentatadmissionaretrospectivestudy AT wenxiangxu developmentandvalidationofmachinelearningmodelsforvenousthromboembolismriskassessmentatadmissionaretrospectivestudy AT zedonghao developmentandvalidationofmachinelearningmodelsforvenousthromboembolismriskassessmentatadmissionaretrospectivestudy AT handongma developmentandvalidationofmachinelearningmodelsforvenousthromboembolismriskassessmentatadmissionaretrospectivestudy AT shaodianzhang developmentandvalidationofmachinelearningmodelsforvenousthromboembolismriskassessmentatadmissionaretrospectivestudy AT shaodianzhang developmentandvalidationofmachinelearningmodelsforvenousthromboembolismriskassessmentatadmissionaretrospectivestudy |