Benchmarking explanation methods for mental state decoding with deep learning models
Deep learning (DL) models find increasing application in mental state decoding, where researchers seek to understand the mapping between mental states (e.g., experiencing anger or joy) and brain activity by identifying those spatial and temporal features of brain activity that allow to accurately id...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-06-01
|
| Series: | NeuroImage |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1053811923002550 |
| _version_ | 1797837117909368832 |
|---|---|
| author | Armin W. Thomas Christopher Ré Russell A. Poldrack |
| author_facet | Armin W. Thomas Christopher Ré Russell A. Poldrack |
| author_sort | Armin W. Thomas |
| collection | DOAJ |
| description | Deep learning (DL) models find increasing application in mental state decoding, where researchers seek to understand the mapping between mental states (e.g., experiencing anger or joy) and brain activity by identifying those spatial and temporal features of brain activity that allow to accurately identify (i.e., decode) these states. Once a DL model has been trained to accurately decode a set of mental states, neuroimaging researchers often make use of methods from explainable artificial intelligence research to understand the model’s learned mappings between mental states and brain activity. Here, we benchmark prominent explanation methods in a mental state decoding analysis of multiple functional Magnetic Resonance Imaging (fMRI) datasets. Our findings demonstrate a gradient between two key characteristics of an explanation in mental state decoding, namely, its faithfulness and its alignment with other empirical evidence on the mapping between brain activity and decoded mental state: explanation methods with high explanation faithfulness, which capture the model’s decision process well, generally provide explanations that align less well with other empirical evidence than the explanations of methods with less faithfulness. Based on our findings, we provide guidance for neuroimaging researchers on how to choose an explanation method to gain insight into the mental state decoding decisions of DL models. |
| first_indexed | 2024-04-09T15:20:47Z |
| format | Article |
| id | doaj.art-bb66e917566542cbb5949ed27d6b8661 |
| institution | Directory Open Access Journal |
| issn | 1095-9572 |
| language | English |
| last_indexed | 2024-04-09T15:20:47Z |
| publishDate | 2023-06-01 |
| publisher | Elsevier |
| record_format | Article |
| series | NeuroImage |
| spelling | doaj.art-bb66e917566542cbb5949ed27d6b86612023-04-29T14:47:32ZengElsevierNeuroImage1095-95722023-06-01273120109Benchmarking explanation methods for mental state decoding with deep learning modelsArmin W. Thomas0Christopher Ré1Russell A. Poldrack2Corresponding author.; Stanford Data Science, Stanford University, 450 Serra Mall, 94305, Stanford, USADept. of Computer Science, Stanford University, 450 Serra Mall, 94305, Stanford, USADept. of Psychology, Stanford University, 450 Serra Mall, Stanford, 94305, USADeep learning (DL) models find increasing application in mental state decoding, where researchers seek to understand the mapping between mental states (e.g., experiencing anger or joy) and brain activity by identifying those spatial and temporal features of brain activity that allow to accurately identify (i.e., decode) these states. Once a DL model has been trained to accurately decode a set of mental states, neuroimaging researchers often make use of methods from explainable artificial intelligence research to understand the model’s learned mappings between mental states and brain activity. Here, we benchmark prominent explanation methods in a mental state decoding analysis of multiple functional Magnetic Resonance Imaging (fMRI) datasets. Our findings demonstrate a gradient between two key characteristics of an explanation in mental state decoding, namely, its faithfulness and its alignment with other empirical evidence on the mapping between brain activity and decoded mental state: explanation methods with high explanation faithfulness, which capture the model’s decision process well, generally provide explanations that align less well with other empirical evidence than the explanations of methods with less faithfulness. Based on our findings, we provide guidance for neuroimaging researchers on how to choose an explanation method to gain insight into the mental state decoding decisions of DL models.http://www.sciencedirect.com/science/article/pii/S1053811923002550NeuroimagingMental state decodingDeep learningExplainable AIBenchmark |
| spellingShingle | Armin W. Thomas Christopher Ré Russell A. Poldrack Benchmarking explanation methods for mental state decoding with deep learning models NeuroImage Neuroimaging Mental state decoding Deep learning Explainable AI Benchmark |
| title | Benchmarking explanation methods for mental state decoding with deep learning models |
| title_full | Benchmarking explanation methods for mental state decoding with deep learning models |
| title_fullStr | Benchmarking explanation methods for mental state decoding with deep learning models |
| title_full_unstemmed | Benchmarking explanation methods for mental state decoding with deep learning models |
| title_short | Benchmarking explanation methods for mental state decoding with deep learning models |
| title_sort | benchmarking explanation methods for mental state decoding with deep learning models |
| topic | Neuroimaging Mental state decoding Deep learning Explainable AI Benchmark |
| url | http://www.sciencedirect.com/science/article/pii/S1053811923002550 |
| work_keys_str_mv | AT arminwthomas benchmarkingexplanationmethodsformentalstatedecodingwithdeeplearningmodels AT christopherre benchmarkingexplanationmethodsformentalstatedecodingwithdeeplearningmodels AT russellapoldrack benchmarkingexplanationmethodsformentalstatedecodingwithdeeplearningmodels |