A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study
BackgroundNumerous studies have identified risk factors for physical restraint (PR) use in older adults in long-term care facilities. Nevertheless, there is a lack of predictive tools to identify high-risk individuals. ObjectiveWe aimed to develop machine learning...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2023-04-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2023/1/e43815 |
| _version_ | 1827857416975286272 |
|---|---|
| author | Jun Wang Hongmei Chen Houwei Wang Weichu Liu Daomei Peng Qinghua Zhao Mingzhao Xiao |
| author_facet | Jun Wang Hongmei Chen Houwei Wang Weichu Liu Daomei Peng Qinghua Zhao Mingzhao Xiao |
| author_sort | Jun Wang |
| collection | DOAJ |
| description |
BackgroundNumerous studies have identified risk factors for physical restraint (PR) use in older adults in long-term care facilities. Nevertheless, there is a lack of predictive tools to identify high-risk individuals.
ObjectiveWe aimed to develop machine learning (ML)–based models to predict the risk of PR in older adults.
MethodsThis study conducted a cross-sectional secondary data analysis based on 1026 older adults from 6 long-term care facilities in Chongqing, China, from July 2019 to November 2019. The primary outcome was the use of PR (yes or no), identified by 2 collectors’ direct observation. A total of 15 candidate predictors (older adults’ demographic and clinical factors) that could be commonly and easily collected from clinical practice were used to build 9 independent ML models: Gaussian Naïve Bayesian (GNB), k-nearest neighbor (KNN), decision tree (DT), logistic regression (LR), support vector machine (SVM), random forest (RF), multilayer perceptron (MLP), extreme gradient boosting (XGBoost), and light gradient boosting machine (Lightgbm), as well as stacking ensemble ML. Performance was evaluated using accuracy, precision, recall, an F score, a comprehensive evaluation indicator (CEI) weighed by the above indicators, and the area under the receiver operating characteristic curve (AUC). A net benefit approach using the decision curve analysis (DCA) was performed to evaluate the clinical utility of the best model. Models were tested via 10-fold cross-validation. Feature importance was interpreted using Shapley Additive Explanations (SHAP).
ResultsA total of 1026 older adults (mean 83.5, SD 7.6 years; n=586, 57.1% male older adults) and 265 restrained older adults were included in the study. All ML models performed well, with an AUC above 0.905 and an F score above 0.900. The 2 best independent models are RF (AUC 0.938, 95% CI 0.914-0.947) and SVM (AUC 0.949, 95% CI 0.911-0.953). The DCA demonstrated that the RF model displayed better clinical utility than other models. The stacking model combined with SVM, RF, and MLP performed best with AUC (0.950) and CEI (0.943) values, as well as the DCA curve indicated the best clinical utility. The SHAP plots demonstrated that the significant contributors to model performance were related to cognitive impairment, care dependency, mobility decline, physical agitation, and an indwelling tube.
ConclusionsThe RF and stacking models had high performance and clinical utility. ML prediction models for predicting the probability of PR in older adults could offer clinical screening and decision support, which could help medical staff in the early identification and PR management of older adults. |
| first_indexed | 2024-03-12T12:40:13Z |
| format | Article |
| id | doaj.art-4e8676f1ce584955af8bc2f4c965424f |
| institution | Directory Open Access Journal |
| issn | 1438-8871 |
| language | English |
| last_indexed | 2024-03-12T12:40:13Z |
| publishDate | 2023-04-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj.art-4e8676f1ce584955af8bc2f4c965424f2023-08-28T23:51:28ZengJMIR PublicationsJournal of Medical Internet Research1438-88712023-04-0125e4381510.2196/43815A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning StudyJun Wanghttps://orcid.org/0000-0002-7518-7135Hongmei Chenhttps://orcid.org/0000-0003-2302-2117Houwei Wanghttps://orcid.org/0009-0003-8255-9640Weichu Liuhttps://orcid.org/0009-0002-6204-8758Daomei Penghttps://orcid.org/0009-0009-7411-4207Qinghua Zhaohttps://orcid.org/0000-0002-3115-3128Mingzhao Xiaohttps://orcid.org/0000-0001-6989-5971 BackgroundNumerous studies have identified risk factors for physical restraint (PR) use in older adults in long-term care facilities. Nevertheless, there is a lack of predictive tools to identify high-risk individuals. ObjectiveWe aimed to develop machine learning (ML)–based models to predict the risk of PR in older adults. MethodsThis study conducted a cross-sectional secondary data analysis based on 1026 older adults from 6 long-term care facilities in Chongqing, China, from July 2019 to November 2019. The primary outcome was the use of PR (yes or no), identified by 2 collectors’ direct observation. A total of 15 candidate predictors (older adults’ demographic and clinical factors) that could be commonly and easily collected from clinical practice were used to build 9 independent ML models: Gaussian Naïve Bayesian (GNB), k-nearest neighbor (KNN), decision tree (DT), logistic regression (LR), support vector machine (SVM), random forest (RF), multilayer perceptron (MLP), extreme gradient boosting (XGBoost), and light gradient boosting machine (Lightgbm), as well as stacking ensemble ML. Performance was evaluated using accuracy, precision, recall, an F score, a comprehensive evaluation indicator (CEI) weighed by the above indicators, and the area under the receiver operating characteristic curve (AUC). A net benefit approach using the decision curve analysis (DCA) was performed to evaluate the clinical utility of the best model. Models were tested via 10-fold cross-validation. Feature importance was interpreted using Shapley Additive Explanations (SHAP). ResultsA total of 1026 older adults (mean 83.5, SD 7.6 years; n=586, 57.1% male older adults) and 265 restrained older adults were included in the study. All ML models performed well, with an AUC above 0.905 and an F score above 0.900. The 2 best independent models are RF (AUC 0.938, 95% CI 0.914-0.947) and SVM (AUC 0.949, 95% CI 0.911-0.953). The DCA demonstrated that the RF model displayed better clinical utility than other models. The stacking model combined with SVM, RF, and MLP performed best with AUC (0.950) and CEI (0.943) values, as well as the DCA curve indicated the best clinical utility. The SHAP plots demonstrated that the significant contributors to model performance were related to cognitive impairment, care dependency, mobility decline, physical agitation, and an indwelling tube. ConclusionsThe RF and stacking models had high performance and clinical utility. ML prediction models for predicting the probability of PR in older adults could offer clinical screening and decision support, which could help medical staff in the early identification and PR management of older adults.https://www.jmir.org/2023/1/e43815 |
| spellingShingle | Jun Wang Hongmei Chen Houwei Wang Weichu Liu Daomei Peng Qinghua Zhao Mingzhao Xiao A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study Journal of Medical Internet Research |
| title | A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study |
| title_full | A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study |
| title_fullStr | A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study |
| title_full_unstemmed | A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study |
| title_short | A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study |
| title_sort | risk prediction model for physical restraints among older chinese adults in long term care facilities machine learning study |
| url | https://www.jmir.org/2023/1/e43815 |
| work_keys_str_mv | AT junwang ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT hongmeichen ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT houweiwang ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT weichuliu ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT daomeipeng ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT qinghuazhao ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT mingzhaoxiao ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT junwang riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT hongmeichen riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT houweiwang riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT weichuliu riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT daomeipeng riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT qinghuazhao riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy AT mingzhaoxiao riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy |