Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network
Deep neural networks (DNNs) can accurately decode task-related information from brain activations. However, because of the non-linearity of DNNs, it is generally difficult to explain how and why they assign certain behavioral tasks to given brain activations, either correctly or incorrectly. One of...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-03-01
|
| Series: | Frontiers in Neuroinformatics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fninf.2021.802938/full |
| _version_ | 1818338171034796032 |
|---|---|
| author | Teppei Matsui Teppei Matsui Masato Taki Trung Quang Pham Junichi Chikazoe Junichi Chikazoe Koji Jimura |
| author_facet | Teppei Matsui Teppei Matsui Masato Taki Trung Quang Pham Junichi Chikazoe Junichi Chikazoe Koji Jimura |
| author_sort | Teppei Matsui |
| collection | DOAJ |
| description | Deep neural networks (DNNs) can accurately decode task-related information from brain activations. However, because of the non-linearity of DNNs, it is generally difficult to explain how and why they assign certain behavioral tasks to given brain activations, either correctly or incorrectly. One of the promising approaches for explaining such a black-box system is counterfactual explanation. In this framework, the behavior of a black-box system is explained by comparing real data and realistic synthetic data that are specifically generated such that the black-box system outputs an unreal outcome. The explanation of the system's decision can be explained by directly comparing the real and synthetic data. Recently, by taking advantage of advances in DNN-based image-to-image translation, several studies successfully applied counterfactual explanation to image domains. In principle, the same approach could be used in functional magnetic resonance imaging (fMRI) data. Because fMRI datasets often contain multiple classes (e.g., multiple behavioral tasks), the image-to-image transformation applicable to counterfactual explanation needs to learn mapping among multiple classes simultaneously. Recently, a new generative neural network (StarGAN) that enables image-to-image transformation among multiple classes has been developed. By adapting StarGAN with some modifications, here, we introduce a novel generative DNN (counterfactual activation generator, CAG) that can provide counterfactual explanations for DNN-based classifiers of brain activations. Importantly, CAG can simultaneously handle image transformation among all the seven classes in a publicly available fMRI dataset. Thus, CAG could provide a counterfactual explanation of DNN-based multiclass classifiers of brain activations. Furthermore, iterative applications of CAG were able to enhance and extract subtle spatial brain activity patterns that affected the classifier's decisions. Together, these results demonstrate that the counterfactual explanation based on image-to-image transformation would be a promising approach to understand and extend the current application of DNNs in fMRI analyses. |
| first_indexed | 2024-12-13T15:06:52Z |
| format | Article |
| id | doaj.art-68b631c391a4444eb865ecc95b5ea80c |
| institution | Directory Open Access Journal |
| issn | 1662-5196 |
| language | English |
| last_indexed | 2024-12-13T15:06:52Z |
| publishDate | 2022-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroinformatics |
| spelling | doaj.art-68b631c391a4444eb865ecc95b5ea80c2022-12-21T23:40:58ZengFrontiers Media S.A.Frontiers in Neuroinformatics1662-51962022-03-011510.3389/fninf.2021.802938802938Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial NetworkTeppei Matsui0Teppei Matsui1Masato Taki2Trung Quang Pham3Junichi Chikazoe4Junichi Chikazoe5Koji Jimura6Department of Biology, Okayama University, Okayama, JapanJST-PRESTO, Japan Science and Technology Agency, Tokyo, JapanGraduate School of Artificial Intelligence and Science, Rikkyo University, Tokyo, JapanSupportive Center for Brain Research, National Institute for Physiological Sciences, Okazaki, JapanSupportive Center for Brain Research, National Institute for Physiological Sciences, Okazaki, JapanAraya Inc., Tokyo, JapanDepartment of Biosciences and Informatics, Keio University, Yokohama, JapanDeep neural networks (DNNs) can accurately decode task-related information from brain activations. However, because of the non-linearity of DNNs, it is generally difficult to explain how and why they assign certain behavioral tasks to given brain activations, either correctly or incorrectly. One of the promising approaches for explaining such a black-box system is counterfactual explanation. In this framework, the behavior of a black-box system is explained by comparing real data and realistic synthetic data that are specifically generated such that the black-box system outputs an unreal outcome. The explanation of the system's decision can be explained by directly comparing the real and synthetic data. Recently, by taking advantage of advances in DNN-based image-to-image translation, several studies successfully applied counterfactual explanation to image domains. In principle, the same approach could be used in functional magnetic resonance imaging (fMRI) data. Because fMRI datasets often contain multiple classes (e.g., multiple behavioral tasks), the image-to-image transformation applicable to counterfactual explanation needs to learn mapping among multiple classes simultaneously. Recently, a new generative neural network (StarGAN) that enables image-to-image transformation among multiple classes has been developed. By adapting StarGAN with some modifications, here, we introduce a novel generative DNN (counterfactual activation generator, CAG) that can provide counterfactual explanations for DNN-based classifiers of brain activations. Importantly, CAG can simultaneously handle image transformation among all the seven classes in a publicly available fMRI dataset. Thus, CAG could provide a counterfactual explanation of DNN-based multiclass classifiers of brain activations. Furthermore, iterative applications of CAG were able to enhance and extract subtle spatial brain activity patterns that affected the classifier's decisions. Together, these results demonstrate that the counterfactual explanation based on image-to-image transformation would be a promising approach to understand and extend the current application of DNNs in fMRI analyses.https://www.frontiersin.org/articles/10.3389/fninf.2021.802938/fullfMRIdeep learningexplainable AIdecodinggenerative neural networkcounterfactual explanation |
| spellingShingle | Teppei Matsui Teppei Matsui Masato Taki Trung Quang Pham Junichi Chikazoe Junichi Chikazoe Koji Jimura Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network Frontiers in Neuroinformatics fMRI deep learning explainable AI decoding generative neural network counterfactual explanation |
| title | Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network |
| title_full | Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network |
| title_fullStr | Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network |
| title_full_unstemmed | Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network |
| title_short | Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network |
| title_sort | counterfactual explanation of brain activity classifiers using image to image transfer by generative adversarial network |
| topic | fMRI deep learning explainable AI decoding generative neural network counterfactual explanation |
| url | https://www.frontiersin.org/articles/10.3389/fninf.2021.802938/full |
| work_keys_str_mv | AT teppeimatsui counterfactualexplanationofbrainactivityclassifiersusingimagetoimagetransferbygenerativeadversarialnetwork AT teppeimatsui counterfactualexplanationofbrainactivityclassifiersusingimagetoimagetransferbygenerativeadversarialnetwork AT masatotaki counterfactualexplanationofbrainactivityclassifiersusingimagetoimagetransferbygenerativeadversarialnetwork AT trungquangpham counterfactualexplanationofbrainactivityclassifiersusingimagetoimagetransferbygenerativeadversarialnetwork AT junichichikazoe counterfactualexplanationofbrainactivityclassifiersusingimagetoimagetransferbygenerativeadversarialnetwork AT junichichikazoe counterfactualexplanationofbrainactivityclassifiersusingimagetoimagetransferbygenerativeadversarialnetwork AT kojijimura counterfactualexplanationofbrainactivityclassifiersusingimagetoimagetransferbygenerativeadversarialnetwork |