Explainability of random survival forests in predicting conversion risk from mild cognitive impairment to Alzheimer’s disease
Abstract Random Survival Forests (RSF) has recently showed better performance than statistical survival methods as Cox proportional hazard (CPH) in predicting conversion risk from mild cognitive impairment (MCI) to Alzheimer’s disease (AD). However, RSF application in real-world clinical setting is...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2023-11-01
|
| Series: | Brain Informatics |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40708-023-00211-w |
| _version_ | 1797555599185018880 |
|---|---|
| author | Alessia Sarica Federica Aracri Maria Giovanna Bianco Fulvia Arcuri Andrea Quattrone Aldo Quattrone for the Alzheimer’s Disease Neuroimaging Initiative |
| author_facet | Alessia Sarica Federica Aracri Maria Giovanna Bianco Fulvia Arcuri Andrea Quattrone Aldo Quattrone for the Alzheimer’s Disease Neuroimaging Initiative |
| author_sort | Alessia Sarica |
| collection | DOAJ |
| description | Abstract Random Survival Forests (RSF) has recently showed better performance than statistical survival methods as Cox proportional hazard (CPH) in predicting conversion risk from mild cognitive impairment (MCI) to Alzheimer’s disease (AD). However, RSF application in real-world clinical setting is still limited due to its black-box nature. For this reason, we aimed at providing a comprehensive study of RSF explainability with SHapley Additive exPlanations (SHAP) on biomarkers of stable and progressive patients (sMCI and pMCI) from Alzheimer’s Disease Neuroimaging Initiative. We evaluated three global explanations—RSF feature importance, permutation importance and SHAP importance—and we quantitatively compared them with Rank-Biased Overlap (RBO). Moreover, we assessed whether multicollinearity among variables may perturb SHAP outcome. Lastly, we stratified pMCI test patients in high, medium and low risk grade, to investigate individual SHAP explanation of one pMCI patient per risk group. We confirmed that RSF had higher accuracy (0.890) than CPH (0.819), and its stability and robustness was demonstrated by high overlap (RBO > 90%) between feature rankings within first eight features. SHAP local explanations with and without correlated variables had no substantial difference, showing that multicollinearity did not alter the model. FDG, ABETA42 and HCI were the first important features in global explanations, with the highest contribution also in local explanation. FAQ, mPACCdigit, mPACCtrailsB and RAVLT immediate had the highest influence among all clinical and neuropsychological assessments in increasing progression risk, as particularly evident in pMCI patients’ individual explanation. In conclusion, our findings suggest that RSF represents a useful tool to support clinicians in estimating conversion-to-AD risk and that SHAP explainer boosts its clinical utility with intelligible and interpretable individual outcomes that highlights key features associated with AD prognosis. Graphical Abstract |
| first_indexed | 2024-03-10T16:49:46Z |
| format | Article |
| id | doaj.art-5de38803ca9d452490d257dd7a8b7cd3 |
| institution | Directory Open Access Journal |
| issn | 2198-4018 2198-4026 |
| language | English |
| last_indexed | 2024-03-10T16:49:46Z |
| publishDate | 2023-11-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Brain Informatics |
| spelling | doaj.art-5de38803ca9d452490d257dd7a8b7cd32023-11-20T11:21:17ZengSpringerOpenBrain Informatics2198-40182198-40262023-11-0110111710.1186/s40708-023-00211-wExplainability of random survival forests in predicting conversion risk from mild cognitive impairment to Alzheimer’s diseaseAlessia Sarica0Federica Aracri1Maria Giovanna Bianco2Fulvia Arcuri3Andrea Quattrone4Aldo Quattrone5for the Alzheimer’s Disease Neuroimaging InitiativeNeuroscience Research Center, Department of Medical and Surgical Sciences, Magna Graecia UniversityNeuroscience Research Center, Department of Medical and Surgical Sciences, Magna Graecia UniversityNeuroscience Research Center, Department of Medical and Surgical Sciences, Magna Graecia UniversityNeuroscience Research Center, Department of Medical and Surgical Sciences, Magna Graecia UniversityNeuroscience Research Center, Department of Medical and Surgical Sciences, Magna Graecia UniversityNeuroscience Research Center, Department of Medical and Surgical Sciences, Magna Graecia UniversityAbstract Random Survival Forests (RSF) has recently showed better performance than statistical survival methods as Cox proportional hazard (CPH) in predicting conversion risk from mild cognitive impairment (MCI) to Alzheimer’s disease (AD). However, RSF application in real-world clinical setting is still limited due to its black-box nature. For this reason, we aimed at providing a comprehensive study of RSF explainability with SHapley Additive exPlanations (SHAP) on biomarkers of stable and progressive patients (sMCI and pMCI) from Alzheimer’s Disease Neuroimaging Initiative. We evaluated three global explanations—RSF feature importance, permutation importance and SHAP importance—and we quantitatively compared them with Rank-Biased Overlap (RBO). Moreover, we assessed whether multicollinearity among variables may perturb SHAP outcome. Lastly, we stratified pMCI test patients in high, medium and low risk grade, to investigate individual SHAP explanation of one pMCI patient per risk group. We confirmed that RSF had higher accuracy (0.890) than CPH (0.819), and its stability and robustness was demonstrated by high overlap (RBO > 90%) between feature rankings within first eight features. SHAP local explanations with and without correlated variables had no substantial difference, showing that multicollinearity did not alter the model. FDG, ABETA42 and HCI were the first important features in global explanations, with the highest contribution also in local explanation. FAQ, mPACCdigit, mPACCtrailsB and RAVLT immediate had the highest influence among all clinical and neuropsychological assessments in increasing progression risk, as particularly evident in pMCI patients’ individual explanation. In conclusion, our findings suggest that RSF represents a useful tool to support clinicians in estimating conversion-to-AD risk and that SHAP explainer boosts its clinical utility with intelligible and interpretable individual outcomes that highlights key features associated with AD prognosis. Graphical Abstracthttps://doi.org/10.1186/s40708-023-00211-wSurvival analysisCox proportional hazardRandom Survival ForestsMachine learningMCI conversionAD progression |
| spellingShingle | Alessia Sarica Federica Aracri Maria Giovanna Bianco Fulvia Arcuri Andrea Quattrone Aldo Quattrone for the Alzheimer’s Disease Neuroimaging Initiative Explainability of random survival forests in predicting conversion risk from mild cognitive impairment to Alzheimer’s disease Brain Informatics Survival analysis Cox proportional hazard Random Survival Forests Machine learning MCI conversion AD progression |
| title | Explainability of random survival forests in predicting conversion risk from mild cognitive impairment to Alzheimer’s disease |
| title_full | Explainability of random survival forests in predicting conversion risk from mild cognitive impairment to Alzheimer’s disease |
| title_fullStr | Explainability of random survival forests in predicting conversion risk from mild cognitive impairment to Alzheimer’s disease |
| title_full_unstemmed | Explainability of random survival forests in predicting conversion risk from mild cognitive impairment to Alzheimer’s disease |
| title_short | Explainability of random survival forests in predicting conversion risk from mild cognitive impairment to Alzheimer’s disease |
| title_sort | explainability of random survival forests in predicting conversion risk from mild cognitive impairment to alzheimer s disease |
| topic | Survival analysis Cox proportional hazard Random Survival Forests Machine learning MCI conversion AD progression |
| url | https://doi.org/10.1186/s40708-023-00211-w |
| work_keys_str_mv | AT alessiasarica explainabilityofrandomsurvivalforestsinpredictingconversionriskfrommildcognitiveimpairmenttoalzheimersdisease AT federicaaracri explainabilityofrandomsurvivalforestsinpredictingconversionriskfrommildcognitiveimpairmenttoalzheimersdisease AT mariagiovannabianco explainabilityofrandomsurvivalforestsinpredictingconversionriskfrommildcognitiveimpairmenttoalzheimersdisease AT fulviaarcuri explainabilityofrandomsurvivalforestsinpredictingconversionriskfrommildcognitiveimpairmenttoalzheimersdisease AT andreaquattrone explainabilityofrandomsurvivalforestsinpredictingconversionriskfrommildcognitiveimpairmenttoalzheimersdisease AT aldoquattrone explainabilityofrandomsurvivalforestsinpredictingconversionriskfrommildcognitiveimpairmenttoalzheimersdisease AT forthealzheimersdiseaseneuroimaginginitiative explainabilityofrandomsurvivalforestsinpredictingconversionriskfrommildcognitiveimpairmenttoalzheimersdisease |