An Artificial Neural Network Prediction Model for Posttraumatic Epilepsy: Retrospective Cohort Study
BackgroundPosttraumatic epilepsy (PTE) is a common sequela after traumatic brain injury (TBI), and identifying high-risk patients with PTE is necessary for their better treatment. Although artificial neural network (ANN) prediction models have been reported and are superior t...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2021-08-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2021/8/e25090 |
| _version_ | 1797735814345523200 |
|---|---|
| author | Xueping Wang Jie Zhong Ting Lei Deng Chen Haijiao Wang Lina Zhu Shanshan Chu Ling Liu |
| author_facet | Xueping Wang Jie Zhong Ting Lei Deng Chen Haijiao Wang Lina Zhu Shanshan Chu Ling Liu |
| author_sort | Xueping Wang |
| collection | DOAJ |
| description |
BackgroundPosttraumatic epilepsy (PTE) is a common sequela after traumatic brain injury (TBI), and identifying high-risk patients with PTE is necessary for their better treatment. Although artificial neural network (ANN) prediction models have been reported and are superior to traditional models, the ANN prediction model for PTE is lacking.
ObjectiveWe aim to train and validate an ANN model to anticipate the risks of PTE.
MethodsThe training cohort was TBI patients registered at West China Hospital. We used a 5-fold cross-validation approach to train and test the ANN model to avoid overfitting; 21 independent variables were used as input neurons in the ANN models, using a back-propagation algorithm to minimize the loss function. Finally, we obtained sensitivity, specificity, and accuracy of each ANN model from the 5 rounds of cross-validation and compared the accuracy with a nomogram prediction model built in our previous work based on the same population. In addition, we evaluated the performance of the model using patients registered at Chengdu Shang Jin Nan Fu Hospital (testing cohort 1) and Sichuan Provincial People’s Hospital (testing cohort 2) between January 1, 2013, and March 1, 2015.
ResultsFor the training cohort, we enrolled 1301 TBI patients from January 1, 2011, to December 31, 2017. The prevalence of PTE was 12.8% (166/1301, 95% CI 10.9%-14.6%). Of the TBI patients registered in testing cohort 1, PTE prevalence was 10.5% (44/421, 95% CI 7.5%-13.4%). Of the TBI patients registered in testing cohort 2, PTE prevalence was 6.1% (25/413, 95% CI 3.7%-8.4%). The results of the ANN model show that, the area under the receiver operating characteristic curve in the training cohort was 0.907 (95% CI 0.889-0.924), testing cohort 1 was 0.867 (95% CI 0.842-0.893), and testing cohort 2 was 0.859 (95% CI 0.826-0.890). Second, the average accuracy of the training cohort was 0.557 (95% CI 0.510-0.620), with 0.470 (95% CI 0.414-0.526) in testing cohort 1 and 0.344 (95% CI 0.287-0.401) in testing cohort 2. In addition, sensitivity, specificity, positive predictive values and negative predictors in the training cohort (testing cohort 1 and testing cohort 2) were 0.80 (0.83 and 0.80), 0.86 (0.80 and 0.84), 91% (85% and 78%), and 86% (80% and 83%), respectively. When calibrating this ANN model, Brier scored 0.121 in testing cohort 1 and 0.127 in testing cohort 2. Compared with the nomogram model, the ANN prediction model had a higher accuracy (P=.01).
ConclusionsThis study shows that the ANN model can predict the risk of PTE and is superior to the risk estimated based on traditional statistical methods. However, the calibration of the model is a bit poor, and we need to calibrate it on a large sample size set and further improve the model. |
| first_indexed | 2024-03-12T13:03:37Z |
| format | Article |
| id | doaj.art-8c48321213cb41bbb913034dfbc999bc |
| institution | Directory Open Access Journal |
| issn | 1438-8871 |
| language | English |
| last_indexed | 2024-03-12T13:03:37Z |
| publishDate | 2021-08-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj.art-8c48321213cb41bbb913034dfbc999bc2023-08-28T18:32:49ZengJMIR PublicationsJournal of Medical Internet Research1438-88712021-08-01238e2509010.2196/25090An Artificial Neural Network Prediction Model for Posttraumatic Epilepsy: Retrospective Cohort StudyXueping Wanghttps://orcid.org/0000-0002-4176-9786Jie Zhonghttps://orcid.org/0000-0003-2957-1328Ting Leihttps://orcid.org/0000-0002-2222-8861Deng Chenhttps://orcid.org/0000-0002-1073-5604Haijiao Wanghttps://orcid.org/0000-0003-2209-2455Lina Zhuhttps://orcid.org/0000-0003-3671-143XShanshan Chuhttps://orcid.org/0000-0003-2979-9234Ling Liuhttps://orcid.org/0000-0001-7588-6086 BackgroundPosttraumatic epilepsy (PTE) is a common sequela after traumatic brain injury (TBI), and identifying high-risk patients with PTE is necessary for their better treatment. Although artificial neural network (ANN) prediction models have been reported and are superior to traditional models, the ANN prediction model for PTE is lacking. ObjectiveWe aim to train and validate an ANN model to anticipate the risks of PTE. MethodsThe training cohort was TBI patients registered at West China Hospital. We used a 5-fold cross-validation approach to train and test the ANN model to avoid overfitting; 21 independent variables were used as input neurons in the ANN models, using a back-propagation algorithm to minimize the loss function. Finally, we obtained sensitivity, specificity, and accuracy of each ANN model from the 5 rounds of cross-validation and compared the accuracy with a nomogram prediction model built in our previous work based on the same population. In addition, we evaluated the performance of the model using patients registered at Chengdu Shang Jin Nan Fu Hospital (testing cohort 1) and Sichuan Provincial People’s Hospital (testing cohort 2) between January 1, 2013, and March 1, 2015. ResultsFor the training cohort, we enrolled 1301 TBI patients from January 1, 2011, to December 31, 2017. The prevalence of PTE was 12.8% (166/1301, 95% CI 10.9%-14.6%). Of the TBI patients registered in testing cohort 1, PTE prevalence was 10.5% (44/421, 95% CI 7.5%-13.4%). Of the TBI patients registered in testing cohort 2, PTE prevalence was 6.1% (25/413, 95% CI 3.7%-8.4%). The results of the ANN model show that, the area under the receiver operating characteristic curve in the training cohort was 0.907 (95% CI 0.889-0.924), testing cohort 1 was 0.867 (95% CI 0.842-0.893), and testing cohort 2 was 0.859 (95% CI 0.826-0.890). Second, the average accuracy of the training cohort was 0.557 (95% CI 0.510-0.620), with 0.470 (95% CI 0.414-0.526) in testing cohort 1 and 0.344 (95% CI 0.287-0.401) in testing cohort 2. In addition, sensitivity, specificity, positive predictive values and negative predictors in the training cohort (testing cohort 1 and testing cohort 2) were 0.80 (0.83 and 0.80), 0.86 (0.80 and 0.84), 91% (85% and 78%), and 86% (80% and 83%), respectively. When calibrating this ANN model, Brier scored 0.121 in testing cohort 1 and 0.127 in testing cohort 2. Compared with the nomogram model, the ANN prediction model had a higher accuracy (P=.01). ConclusionsThis study shows that the ANN model can predict the risk of PTE and is superior to the risk estimated based on traditional statistical methods. However, the calibration of the model is a bit poor, and we need to calibrate it on a large sample size set and further improve the model.https://www.jmir.org/2021/8/e25090 |
| spellingShingle | Xueping Wang Jie Zhong Ting Lei Deng Chen Haijiao Wang Lina Zhu Shanshan Chu Ling Liu An Artificial Neural Network Prediction Model for Posttraumatic Epilepsy: Retrospective Cohort Study Journal of Medical Internet Research |
| title | An Artificial Neural Network Prediction Model for Posttraumatic Epilepsy: Retrospective Cohort Study |
| title_full | An Artificial Neural Network Prediction Model for Posttraumatic Epilepsy: Retrospective Cohort Study |
| title_fullStr | An Artificial Neural Network Prediction Model for Posttraumatic Epilepsy: Retrospective Cohort Study |
| title_full_unstemmed | An Artificial Neural Network Prediction Model for Posttraumatic Epilepsy: Retrospective Cohort Study |
| title_short | An Artificial Neural Network Prediction Model for Posttraumatic Epilepsy: Retrospective Cohort Study |
| title_sort | artificial neural network prediction model for posttraumatic epilepsy retrospective cohort study |
| url | https://www.jmir.org/2021/8/e25090 |
| work_keys_str_mv | AT xuepingwang anartificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT jiezhong anartificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT tinglei anartificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT dengchen anartificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT haijiaowang anartificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT linazhu anartificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT shanshanchu anartificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT lingliu anartificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT xuepingwang artificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT jiezhong artificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT tinglei artificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT dengchen artificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT haijiaowang artificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT linazhu artificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT shanshanchu artificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy AT lingliu artificialneuralnetworkpredictionmodelforposttraumaticepilepsyretrospectivecohortstudy |