Prediction Model of Osteonecrosis of the Femoral Head After Femoral Neck Fracture: Machine Learning–Based Development and Validation Study
BackgroundThe absolute number of femoral neck fractures (FNFs) is increasing; however, the prediction of traumatic femoral head necrosis remains difficult. Machine learning algorithms have the potential to be superior to traditional prediction methods for the prediction of tr...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2021-11-01
|
| Series: | JMIR Medical Informatics |
| Online Access: | https://medinform.jmir.org/2021/11/e30079 |
| _version_ | 1797735568137781248 |
|---|---|
| author | Huan Wang Wei Wu Chunxia Han Jiaqi Zheng Xinyu Cai Shimin Chang Junlong Shi Nan Xu Zisheng Ai |
| author_facet | Huan Wang Wei Wu Chunxia Han Jiaqi Zheng Xinyu Cai Shimin Chang Junlong Shi Nan Xu Zisheng Ai |
| author_sort | Huan Wang |
| collection | DOAJ |
| description |
BackgroundThe absolute number of femoral neck fractures (FNFs) is increasing; however, the prediction of traumatic femoral head necrosis remains difficult. Machine learning algorithms have the potential to be superior to traditional prediction methods for the prediction of traumatic femoral head necrosis.
ObjectiveThe aim of this study is to use machine learning to construct a model for the analysis of risk factors and prediction of osteonecrosis of the femoral head (ONFH) in patients with FNF after internal fixation.
MethodsWe retrospectively collected preoperative, intraoperative, and postoperative clinical data of patients with FNF in 4 hospitals in Shanghai and followed up the patients for more than 2.5 years. A total of 259 patients with 43 variables were included in the study. The data were randomly divided into a training set (181/259, 69.8%) and a validation set (78/259, 30.1%). External data (n=376) were obtained from a retrospective cohort study of patients with FNF in 3 other hospitals. Least absolute shrinkage and selection operator regression and the support vector machine algorithm were used for variable selection. Logistic regression, random forest, support vector machine, and eXtreme Gradient Boosting (XGBoost) were used to develop the model on the training set. The validation set was used to tune the model hyperparameters to determine the final prediction model, and the external data were used to compare and evaluate the model performance. We compared the accuracy, discrimination, and calibration of the models to identify the best machine learning algorithm for predicting ONFH. Shapley additive explanations and local interpretable model-agnostic explanations were used to determine the interpretability of the black box model.
ResultsA total of 11 variables were selected for the models. The XGBoost model performed best on the validation set and external data. The accuracy, sensitivity, and area under the receiver operating characteristic curve of the model on the validation set were 0.987, 0.929, and 0.992, respectively. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve of the model on the external data were 0.907, 0.807, 0.935, and 0.933, respectively, and the log-loss was 0.279. The calibration curve demonstrated good agreement between the predicted probability and actual risk. The interpretability of the features and individual predictions were realized using the Shapley additive explanations and local interpretable model-agnostic explanations algorithms. In addition, the XGBoost model was translated into a self-made web-based risk calculator to estimate an individual’s probability of ONFH.
ConclusionsMachine learning performs well in predicting ONFH after internal fixation of FNF. The 6-variable XGBoost model predicted the risk of ONFH well and had good generalization ability on the external data, which can be used for the clinical prediction of ONFH after internal fixation of FNF. |
| first_indexed | 2024-03-12T13:00:57Z |
| format | Article |
| id | doaj.art-e06f79633e3c4d32a2b84140653d4529 |
| institution | Directory Open Access Journal |
| issn | 2291-9694 |
| language | English |
| last_indexed | 2024-03-12T13:00:57Z |
| publishDate | 2021-11-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Medical Informatics |
| spelling | doaj.art-e06f79633e3c4d32a2b84140653d45292023-08-28T19:48:43ZengJMIR PublicationsJMIR Medical Informatics2291-96942021-11-01911e3007910.2196/30079Prediction Model of Osteonecrosis of the Femoral Head After Femoral Neck Fracture: Machine Learning–Based Development and Validation StudyHuan Wanghttps://orcid.org/0000-0002-6771-4564Wei Wuhttps://orcid.org/0000-0002-9409-3855Chunxia Hanhttps://orcid.org/0000-0003-4097-083XJiaqi Zhenghttps://orcid.org/0000-0002-8556-6593Xinyu Caihttps://orcid.org/0000-0002-0979-8763Shimin Changhttps://orcid.org/0000-0001-7186-5851Junlong Shihttps://orcid.org/0000-0003-4457-9600Nan Xuhttps://orcid.org/0000-0002-8525-3722Zisheng Aihttps://orcid.org/0000-0002-6264-0965 BackgroundThe absolute number of femoral neck fractures (FNFs) is increasing; however, the prediction of traumatic femoral head necrosis remains difficult. Machine learning algorithms have the potential to be superior to traditional prediction methods for the prediction of traumatic femoral head necrosis. ObjectiveThe aim of this study is to use machine learning to construct a model for the analysis of risk factors and prediction of osteonecrosis of the femoral head (ONFH) in patients with FNF after internal fixation. MethodsWe retrospectively collected preoperative, intraoperative, and postoperative clinical data of patients with FNF in 4 hospitals in Shanghai and followed up the patients for more than 2.5 years. A total of 259 patients with 43 variables were included in the study. The data were randomly divided into a training set (181/259, 69.8%) and a validation set (78/259, 30.1%). External data (n=376) were obtained from a retrospective cohort study of patients with FNF in 3 other hospitals. Least absolute shrinkage and selection operator regression and the support vector machine algorithm were used for variable selection. Logistic regression, random forest, support vector machine, and eXtreme Gradient Boosting (XGBoost) were used to develop the model on the training set. The validation set was used to tune the model hyperparameters to determine the final prediction model, and the external data were used to compare and evaluate the model performance. We compared the accuracy, discrimination, and calibration of the models to identify the best machine learning algorithm for predicting ONFH. Shapley additive explanations and local interpretable model-agnostic explanations were used to determine the interpretability of the black box model. ResultsA total of 11 variables were selected for the models. The XGBoost model performed best on the validation set and external data. The accuracy, sensitivity, and area under the receiver operating characteristic curve of the model on the validation set were 0.987, 0.929, and 0.992, respectively. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve of the model on the external data were 0.907, 0.807, 0.935, and 0.933, respectively, and the log-loss was 0.279. The calibration curve demonstrated good agreement between the predicted probability and actual risk. The interpretability of the features and individual predictions were realized using the Shapley additive explanations and local interpretable model-agnostic explanations algorithms. In addition, the XGBoost model was translated into a self-made web-based risk calculator to estimate an individual’s probability of ONFH. ConclusionsMachine learning performs well in predicting ONFH after internal fixation of FNF. The 6-variable XGBoost model predicted the risk of ONFH well and had good generalization ability on the external data, which can be used for the clinical prediction of ONFH after internal fixation of FNF.https://medinform.jmir.org/2021/11/e30079 |
| spellingShingle | Huan Wang Wei Wu Chunxia Han Jiaqi Zheng Xinyu Cai Shimin Chang Junlong Shi Nan Xu Zisheng Ai Prediction Model of Osteonecrosis of the Femoral Head After Femoral Neck Fracture: Machine Learning–Based Development and Validation Study JMIR Medical Informatics |
| title | Prediction Model of Osteonecrosis of the Femoral Head After Femoral Neck Fracture: Machine Learning–Based Development and Validation Study |
| title_full | Prediction Model of Osteonecrosis of the Femoral Head After Femoral Neck Fracture: Machine Learning–Based Development and Validation Study |
| title_fullStr | Prediction Model of Osteonecrosis of the Femoral Head After Femoral Neck Fracture: Machine Learning–Based Development and Validation Study |
| title_full_unstemmed | Prediction Model of Osteonecrosis of the Femoral Head After Femoral Neck Fracture: Machine Learning–Based Development and Validation Study |
| title_short | Prediction Model of Osteonecrosis of the Femoral Head After Femoral Neck Fracture: Machine Learning–Based Development and Validation Study |
| title_sort | prediction model of osteonecrosis of the femoral head after femoral neck fracture machine learning based development and validation study |
| url | https://medinform.jmir.org/2021/11/e30079 |
| work_keys_str_mv | AT huanwang predictionmodelofosteonecrosisofthefemoralheadafterfemoralneckfracturemachinelearningbaseddevelopmentandvalidationstudy AT weiwu predictionmodelofosteonecrosisofthefemoralheadafterfemoralneckfracturemachinelearningbaseddevelopmentandvalidationstudy AT chunxiahan predictionmodelofosteonecrosisofthefemoralheadafterfemoralneckfracturemachinelearningbaseddevelopmentandvalidationstudy AT jiaqizheng predictionmodelofosteonecrosisofthefemoralheadafterfemoralneckfracturemachinelearningbaseddevelopmentandvalidationstudy AT xinyucai predictionmodelofosteonecrosisofthefemoralheadafterfemoralneckfracturemachinelearningbaseddevelopmentandvalidationstudy AT shiminchang predictionmodelofosteonecrosisofthefemoralheadafterfemoralneckfracturemachinelearningbaseddevelopmentandvalidationstudy AT junlongshi predictionmodelofosteonecrosisofthefemoralheadafterfemoralneckfracturemachinelearningbaseddevelopmentandvalidationstudy AT nanxu predictionmodelofosteonecrosisofthefemoralheadafterfemoralneckfracturemachinelearningbaseddevelopmentandvalidationstudy AT zishengai predictionmodelofosteonecrosisofthefemoralheadafterfemoralneckfracturemachinelearningbaseddevelopmentandvalidationstudy |