XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s disease
Abstract Background Due to the class imbalance issue faced when Alzheimer’s disease (AD) develops from normal cognition (NC) to mild cognitive impairment (MCI), present clinical practice is met with challenges regarding the auxiliary diagnosis of AD using machine learning (ML). This leads to low dia...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-07-01
|
| Series: | BMC Medical Informatics and Decision Making |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12911-023-02238-9 |
| _version_ | 1797769421725368320 |
|---|---|
| author | Fuliang Yi Hui Yang Durong Chen Yao Qin Hongjuan Han Jing Cui Wenlin Bai Yifei Ma Rong Zhang Hongmei Yu |
| author_facet | Fuliang Yi Hui Yang Durong Chen Yao Qin Hongjuan Han Jing Cui Wenlin Bai Yifei Ma Rong Zhang Hongmei Yu |
| author_sort | Fuliang Yi |
| collection | DOAJ |
| description | Abstract Background Due to the class imbalance issue faced when Alzheimer’s disease (AD) develops from normal cognition (NC) to mild cognitive impairment (MCI), present clinical practice is met with challenges regarding the auxiliary diagnosis of AD using machine learning (ML). This leads to low diagnosis performance. We aimed to construct an interpretable framework, extreme gradient boosting-Shapley additive explanations (XGBoost-SHAP), to handle the imbalance among different AD progression statuses at the algorithmic level. We also sought to achieve multiclassification of NC, MCI, and AD. Methods We obtained patient data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database, including clinical information, neuropsychological test results, neuroimaging-derived biomarkers, and APOE-ε4 gene statuses. First, three feature selection algorithms were applied, and they were then included in the XGBoost algorithm. Due to the imbalance among the three classes, we changed the sample weight distribution to achieve multiclassification of NC, MCI, and AD. Then, the SHAP method was linked to XGBoost to form an interpretable framework. This framework utilized attribution ideas that quantified the impacts of model predictions into numerical values and analysed them based on their directions and sizes. Subsequently, the top 10 features (optimal subset) were used to simplify the clinical decision-making process, and their performance was compared with that of a random forest (RF), Bagging, AdaBoost, and a naive Bayes (NB) classifier. Finally, the National Alzheimer’s Coordinating Center (NACC) dataset was employed to assess the impact path consistency of the features within the optimal subset. Results Compared to the RF, Bagging, AdaBoost, NB and XGBoost (unweighted), the interpretable framework had higher classification performance with accuracy improvements of 0.74%, 0.74%, 1.46%, 13.18%, and 0.83%, respectively. The framework achieved high sensitivity (81.21%/74.85%), specificity (92.18%/89.86%), accuracy (87.57%/80.52%), area under the receiver operating characteristic curve (AUC) (0.91/0.88), positive clinical utility index (0.71/0.56), and negative clinical utility index (0.75/0.68) on the ADNI and NACC datasets, respectively. In the ADNI dataset, the top 10 features were found to have varying associations with the risk of AD onset based on their SHAP values. Specifically, the higher SHAP values of CDRSB, ADAS13, ADAS11, ventricle volume, ADASQ4, and FAQ were associated with higher risks of AD onset. Conversely, the higher SHAP values of LDELTOTAL, mPACCdigit, RAVLT_immediate, and MMSE were associated with lower risks of AD onset. Similar results were found for the NACC dataset. Conclusions The proposed interpretable framework contributes to achieving excellent performance in imbalanced AD multiclassification tasks and provides scientific guidance (optimal subset) for clinical decision-making, thereby facilitating disease management and offering new research ideas for optimizing AD prevention and treatment programs. |
| first_indexed | 2024-03-12T21:08:47Z |
| format | Article |
| id | doaj.art-2f7158995ba544838dc87c3f0dbde75f |
| institution | Directory Open Access Journal |
| issn | 1472-6947 |
| language | English |
| last_indexed | 2024-03-12T21:08:47Z |
| publishDate | 2023-07-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Informatics and Decision Making |
| spelling | doaj.art-2f7158995ba544838dc87c3f0dbde75f2023-07-30T11:17:17ZengBMCBMC Medical Informatics and Decision Making1472-69472023-07-0123111410.1186/s12911-023-02238-9XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s diseaseFuliang Yi0Hui Yang1Durong Chen2Yao Qin3Hongjuan Han4Jing Cui5Wenlin Bai6Yifei Ma7Rong Zhang8Hongmei Yu9Department of Health Statistics, School of Public Health, Shanxi Medical UniversityDepartment of Health Statistics, School of Public Health, Shanxi Medical UniversityDepartment of Health Statistics, School of Public Health, Shanxi Medical UniversityDepartment of Health Statistics, School of Public Health, Shanxi Medical UniversityDepartment of Health Statistics, School of Public Health, Shanxi Medical UniversityDepartment of Health Statistics, School of Public Health, Shanxi Medical UniversityDepartment of Health Statistics, School of Public Health, Shanxi Medical UniversityDepartment of Health Statistics, School of Public Health, Shanxi Medical UniversityDepartment of Health Statistics, School of Public Health, Shanxi Medical UniversityDepartment of Health Statistics, School of Public Health, Shanxi Medical UniversityAbstract Background Due to the class imbalance issue faced when Alzheimer’s disease (AD) develops from normal cognition (NC) to mild cognitive impairment (MCI), present clinical practice is met with challenges regarding the auxiliary diagnosis of AD using machine learning (ML). This leads to low diagnosis performance. We aimed to construct an interpretable framework, extreme gradient boosting-Shapley additive explanations (XGBoost-SHAP), to handle the imbalance among different AD progression statuses at the algorithmic level. We also sought to achieve multiclassification of NC, MCI, and AD. Methods We obtained patient data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database, including clinical information, neuropsychological test results, neuroimaging-derived biomarkers, and APOE-ε4 gene statuses. First, three feature selection algorithms were applied, and they were then included in the XGBoost algorithm. Due to the imbalance among the three classes, we changed the sample weight distribution to achieve multiclassification of NC, MCI, and AD. Then, the SHAP method was linked to XGBoost to form an interpretable framework. This framework utilized attribution ideas that quantified the impacts of model predictions into numerical values and analysed them based on their directions and sizes. Subsequently, the top 10 features (optimal subset) were used to simplify the clinical decision-making process, and their performance was compared with that of a random forest (RF), Bagging, AdaBoost, and a naive Bayes (NB) classifier. Finally, the National Alzheimer’s Coordinating Center (NACC) dataset was employed to assess the impact path consistency of the features within the optimal subset. Results Compared to the RF, Bagging, AdaBoost, NB and XGBoost (unweighted), the interpretable framework had higher classification performance with accuracy improvements of 0.74%, 0.74%, 1.46%, 13.18%, and 0.83%, respectively. The framework achieved high sensitivity (81.21%/74.85%), specificity (92.18%/89.86%), accuracy (87.57%/80.52%), area under the receiver operating characteristic curve (AUC) (0.91/0.88), positive clinical utility index (0.71/0.56), and negative clinical utility index (0.75/0.68) on the ADNI and NACC datasets, respectively. In the ADNI dataset, the top 10 features were found to have varying associations with the risk of AD onset based on their SHAP values. Specifically, the higher SHAP values of CDRSB, ADAS13, ADAS11, ventricle volume, ADASQ4, and FAQ were associated with higher risks of AD onset. Conversely, the higher SHAP values of LDELTOTAL, mPACCdigit, RAVLT_immediate, and MMSE were associated with lower risks of AD onset. Similar results were found for the NACC dataset. Conclusions The proposed interpretable framework contributes to achieving excellent performance in imbalanced AD multiclassification tasks and provides scientific guidance (optimal subset) for clinical decision-making, thereby facilitating disease management and offering new research ideas for optimizing AD prevention and treatment programs.https://doi.org/10.1186/s12911-023-02238-9Alzheimer’s diseaseMachine learningImbalanced classesMulticlassificationInterpretable frameworkXGBoost-SHAP |
| spellingShingle | Fuliang Yi Hui Yang Durong Chen Yao Qin Hongjuan Han Jing Cui Wenlin Bai Yifei Ma Rong Zhang Hongmei Yu XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s disease BMC Medical Informatics and Decision Making Alzheimer’s disease Machine learning Imbalanced classes Multiclassification Interpretable framework XGBoost-SHAP |
| title | XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s disease |
| title_full | XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s disease |
| title_fullStr | XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s disease |
| title_full_unstemmed | XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s disease |
| title_short | XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s disease |
| title_sort | xgboost shap based interpretable diagnostic framework for alzheimer s disease |
| topic | Alzheimer’s disease Machine learning Imbalanced classes Multiclassification Interpretable framework XGBoost-SHAP |
| url | https://doi.org/10.1186/s12911-023-02238-9 |
| work_keys_str_mv | AT fuliangyi xgboostshapbasedinterpretablediagnosticframeworkforalzheimersdisease AT huiyang xgboostshapbasedinterpretablediagnosticframeworkforalzheimersdisease AT durongchen xgboostshapbasedinterpretablediagnosticframeworkforalzheimersdisease AT yaoqin xgboostshapbasedinterpretablediagnosticframeworkforalzheimersdisease AT hongjuanhan xgboostshapbasedinterpretablediagnosticframeworkforalzheimersdisease AT jingcui xgboostshapbasedinterpretablediagnosticframeworkforalzheimersdisease AT wenlinbai xgboostshapbasedinterpretablediagnosticframeworkforalzheimersdisease AT yifeima xgboostshapbasedinterpretablediagnosticframeworkforalzheimersdisease AT rongzhang xgboostshapbasedinterpretablediagnosticframeworkforalzheimersdisease AT hongmeiyu xgboostshapbasedinterpretablediagnosticframeworkforalzheimersdisease |