Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility Study
Background: The prediction of aneurysm treatment outcomes can help to optimize the treatment strategies. Machine learning (ML) has shown positive results in many clinical areas. However, the development of such models requires expertise in ML, which is not an easy task for surgeons.Objectives: The r...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-11-01
|
| Series: | Frontiers in Neurology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fneur.2021.735142/full |
| _version_ | 1818742232238260224 |
|---|---|
| author | Chubin Ou Chubin Ou Jiahui Liu Yi Qian Winston Chong Dangqi Liu Xuying He Xin Zhang Chuan-Zhi Duan |
| author_facet | Chubin Ou Chubin Ou Jiahui Liu Yi Qian Winston Chong Dangqi Liu Xuying He Xin Zhang Chuan-Zhi Duan |
| author_sort | Chubin Ou |
| collection | DOAJ |
| description | Background: The prediction of aneurysm treatment outcomes can help to optimize the treatment strategies. Machine learning (ML) has shown positive results in many clinical areas. However, the development of such models requires expertise in ML, which is not an easy task for surgeons.Objectives: The recently emerged automated machine learning (AutoML) has shown promise in making ML more accessible to non-computer experts. We aimed to evaluate the feasibility of applying AutoML to develop the ML models for treatment outcome prediction.Methods: The patients with aneurysms treated by endovascular treatment were prospectively recruited from 2016 to 2020. Treatment was considered successful if angiographic complete occlusion was achieved at follow-up. A statistical prediction model was developed using multivariate logistic regression. In addition, two ML models were developed. One was developed manually and the other was developed by AutoML. Three models were compared based on their area under the precision-recall curve (AUPRC) and area under the receiver operating characteristic curve (AUROC).Results: The aneurysm size, stent-assisted coiling (SAC), and posterior circulation were the three significant and independent variables associated with treatment outcome. The statistical model showed an AUPRC of 0.432 and AUROC of 0.745. The conventional manually trained ML model showed an improved AUPRC of 0.545 and AUROC of 0.781. The AutoML derived ML model showed the best performance with AUPRC of 0.632 and AUROC of 0.832, significantly better than the other two models.Conclusions: This study demonstrated the feasibility of using AutoML to develop a high-quality ML model, which may outperform the statistical model and manually derived ML models. AutoML could be a useful tool that makes ML more accessible to the clinical researchers. |
| first_indexed | 2024-12-18T02:09:15Z |
| format | Article |
| id | doaj.art-72a0a387dcfb46f6aad06cd082ba06b4 |
| institution | Directory Open Access Journal |
| issn | 1664-2295 |
| language | English |
| last_indexed | 2024-12-18T02:09:15Z |
| publishDate | 2021-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurology |
| spelling | doaj.art-72a0a387dcfb46f6aad06cd082ba06b42022-12-21T21:24:31ZengFrontiers Media S.A.Frontiers in Neurology1664-22952021-11-011210.3389/fneur.2021.735142735142Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility StudyChubin Ou0Chubin Ou1Jiahui Liu2Yi Qian3Winston Chong4Dangqi Liu5Xuying He6Xin Zhang7Chuan-Zhi Duan8Neurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, ChinaDepartment of Biomedical Sciences, Faculty of Medicine and Health Sciences, Macquarie University, Sydney, NSW, AustraliaNeurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, ChinaNeurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, ChinaMonash Medical Centre, Monash University, Clayton, VIC, AustraliaDepartment of Neurosurgery, The First People's Hospital of Foshan, Foshan, ChinaNeurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, ChinaNeurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, ChinaNeurosurgery Center, Department of Cerebrovascular Surgery, The National Key Clinical Specialty, The Engineering Technology Research Center of Education Ministry of China on Diagnosis and Treatment of Cerebrovascular Disease, Guangdong Provincial Key Laboratory on Brain Function Repair and Regeneration, The Neurosurgery Institute of Guangdong Province, Zhujiang Hospital, Southern Medical University, Guangzhou, ChinaBackground: The prediction of aneurysm treatment outcomes can help to optimize the treatment strategies. Machine learning (ML) has shown positive results in many clinical areas. However, the development of such models requires expertise in ML, which is not an easy task for surgeons.Objectives: The recently emerged automated machine learning (AutoML) has shown promise in making ML more accessible to non-computer experts. We aimed to evaluate the feasibility of applying AutoML to develop the ML models for treatment outcome prediction.Methods: The patients with aneurysms treated by endovascular treatment were prospectively recruited from 2016 to 2020. Treatment was considered successful if angiographic complete occlusion was achieved at follow-up. A statistical prediction model was developed using multivariate logistic regression. In addition, two ML models were developed. One was developed manually and the other was developed by AutoML. Three models were compared based on their area under the precision-recall curve (AUPRC) and area under the receiver operating characteristic curve (AUROC).Results: The aneurysm size, stent-assisted coiling (SAC), and posterior circulation were the three significant and independent variables associated with treatment outcome. The statistical model showed an AUPRC of 0.432 and AUROC of 0.745. The conventional manually trained ML model showed an improved AUPRC of 0.545 and AUROC of 0.781. The AutoML derived ML model showed the best performance with AUPRC of 0.632 and AUROC of 0.832, significantly better than the other two models.Conclusions: This study demonstrated the feasibility of using AutoML to develop a high-quality ML model, which may outperform the statistical model and manually derived ML models. AutoML could be a useful tool that makes ML more accessible to the clinical researchers.https://www.frontiersin.org/articles/10.3389/fneur.2021.735142/fullintracranial aneurysmAutoMLendovascular treatmentmachine learningstroke |
| spellingShingle | Chubin Ou Chubin Ou Jiahui Liu Yi Qian Winston Chong Dangqi Liu Xuying He Xin Zhang Chuan-Zhi Duan Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility Study Frontiers in Neurology intracranial aneurysm AutoML endovascular treatment machine learning stroke |
| title | Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility Study |
| title_full | Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility Study |
| title_fullStr | Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility Study |
| title_full_unstemmed | Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility Study |
| title_short | Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility Study |
| title_sort | automated machine learning model development for intracranial aneurysm treatment outcome prediction a feasibility study |
| topic | intracranial aneurysm AutoML endovascular treatment machine learning stroke |
| url | https://www.frontiersin.org/articles/10.3389/fneur.2021.735142/full |
| work_keys_str_mv | AT chubinou automatedmachinelearningmodeldevelopmentforintracranialaneurysmtreatmentoutcomepredictionafeasibilitystudy AT chubinou automatedmachinelearningmodeldevelopmentforintracranialaneurysmtreatmentoutcomepredictionafeasibilitystudy AT jiahuiliu automatedmachinelearningmodeldevelopmentforintracranialaneurysmtreatmentoutcomepredictionafeasibilitystudy AT yiqian automatedmachinelearningmodeldevelopmentforintracranialaneurysmtreatmentoutcomepredictionafeasibilitystudy AT winstonchong automatedmachinelearningmodeldevelopmentforintracranialaneurysmtreatmentoutcomepredictionafeasibilitystudy AT dangqiliu automatedmachinelearningmodeldevelopmentforintracranialaneurysmtreatmentoutcomepredictionafeasibilitystudy AT xuyinghe automatedmachinelearningmodeldevelopmentforintracranialaneurysmtreatmentoutcomepredictionafeasibilitystudy AT xinzhang automatedmachinelearningmodeldevelopmentforintracranialaneurysmtreatmentoutcomepredictionafeasibilitystudy AT chuanzhiduan automatedmachinelearningmodeldevelopmentforintracranialaneurysmtreatmentoutcomepredictionafeasibilitystudy |