CsAGP: Detecting Alzheimer's disease from multimodal images via dual-transformer with cross-attention and graph pooling
Alzheimer's disease (AD) is a terrible and degenerative disease commonly occurring in the elderly. Early detection can prevent patients from further damage, which is crucial in treating AD. Over the past few decades, it has been demonstrated that neuroimaging can be a critical diagnostic tool f...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-07-01
|
| Series: | Journal of King Saud University: Computer and Information Sciences |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1319157823001726 |
| _version_ | 1797749210856030208 |
|---|---|
| author | Chaosheng Tang Mingyang Wei Junding Sun Shuihua Wang Yudong Zhang |
| author_facet | Chaosheng Tang Mingyang Wei Junding Sun Shuihua Wang Yudong Zhang |
| author_sort | Chaosheng Tang |
| collection | DOAJ |
| description | Alzheimer's disease (AD) is a terrible and degenerative disease commonly occurring in the elderly. Early detection can prevent patients from further damage, which is crucial in treating AD. Over the past few decades, it has been demonstrated that neuroimaging can be a critical diagnostic tool for AD, and the feature fusion of different neuroimaging modalities can enhance diagnostic performance. Most previous studies in multimodal feature fusion have only concatenated the high-level features extracted by neural networks from various neuroimaging images simply. However, a major problem of these studies is overlooking the low-level feature interactions between modalities in the feature extraction stage, resulting in suboptimal performance in AD diagnosis. In this paper, we develop a dual-branch vision transformer with cross-attention and graph pooling, namely CsAGP, which enables multi-level feature interactions between the inputs to learn a shared feature representation. Specifically, we first construct a brand-new cross-attention fusion module (CAFM), which processes MRI and PET images by two independent branches of differing computational complexity. These features are fused merely by the cross-attention mechanism to enhance each other. After that, a concise graph pooling algorithm-based Reshape-Pooling-Reshape (RPR) framework is developed for token selection to reduce token redundancy in the proposed model. Extensive experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) database demonstrated that the suggested method obtains 99.04%, 97.43%, 98.57%, and 98.72% accuracy for the classification of AD vs. CN, AD vs. MCI, CN vs. MCI, and AD vs. CN vs. MCI, respectively. |
| first_indexed | 2024-03-12T16:15:59Z |
| format | Article |
| id | doaj.art-f0411997edf448fa8b47af113b5f0219 |
| institution | Directory Open Access Journal |
| issn | 1319-1578 |
| language | English |
| last_indexed | 2024-03-12T16:15:59Z |
| publishDate | 2023-07-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Journal of King Saud University: Computer and Information Sciences |
| spelling | doaj.art-f0411997edf448fa8b47af113b5f02192023-08-09T04:32:06ZengElsevierJournal of King Saud University: Computer and Information Sciences1319-15782023-07-01357101618CsAGP: Detecting Alzheimer's disease from multimodal images via dual-transformer with cross-attention and graph poolingChaosheng Tang0Mingyang Wei1Junding Sun2Shuihua Wang3Yudong Zhang4School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, Henan 454000, PR ChinaSchool of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, Henan 454000, PR ChinaSchool of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, Henan 454000, PR China; Corresponding authors.School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, Henan 454000, PR China; School of Computing and Mathematical Sciences, University of Leicester, Leicester LE1 7RH, UK; Department of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia; Corresponding authors.School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo, Henan 454000, PR China; School of Computing and Mathematical Sciences, University of Leicester, Leicester LE1 7RH, UK; Department of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia; Corresponding authors.Alzheimer's disease (AD) is a terrible and degenerative disease commonly occurring in the elderly. Early detection can prevent patients from further damage, which is crucial in treating AD. Over the past few decades, it has been demonstrated that neuroimaging can be a critical diagnostic tool for AD, and the feature fusion of different neuroimaging modalities can enhance diagnostic performance. Most previous studies in multimodal feature fusion have only concatenated the high-level features extracted by neural networks from various neuroimaging images simply. However, a major problem of these studies is overlooking the low-level feature interactions between modalities in the feature extraction stage, resulting in suboptimal performance in AD diagnosis. In this paper, we develop a dual-branch vision transformer with cross-attention and graph pooling, namely CsAGP, which enables multi-level feature interactions between the inputs to learn a shared feature representation. Specifically, we first construct a brand-new cross-attention fusion module (CAFM), which processes MRI and PET images by two independent branches of differing computational complexity. These features are fused merely by the cross-attention mechanism to enhance each other. After that, a concise graph pooling algorithm-based Reshape-Pooling-Reshape (RPR) framework is developed for token selection to reduce token redundancy in the proposed model. Extensive experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) database demonstrated that the suggested method obtains 99.04%, 97.43%, 98.57%, and 98.72% accuracy for the classification of AD vs. CN, AD vs. MCI, CN vs. MCI, and AD vs. CN vs. MCI, respectively.http://www.sciencedirect.com/science/article/pii/S1319157823001726Alzheimer's diseaseVision transformerMultimodal image fusionDeep learning |
| spellingShingle | Chaosheng Tang Mingyang Wei Junding Sun Shuihua Wang Yudong Zhang CsAGP: Detecting Alzheimer's disease from multimodal images via dual-transformer with cross-attention and graph pooling Journal of King Saud University: Computer and Information Sciences Alzheimer's disease Vision transformer Multimodal image fusion Deep learning |
| title | CsAGP: Detecting Alzheimer's disease from multimodal images via dual-transformer with cross-attention and graph pooling |
| title_full | CsAGP: Detecting Alzheimer's disease from multimodal images via dual-transformer with cross-attention and graph pooling |
| title_fullStr | CsAGP: Detecting Alzheimer's disease from multimodal images via dual-transformer with cross-attention and graph pooling |
| title_full_unstemmed | CsAGP: Detecting Alzheimer's disease from multimodal images via dual-transformer with cross-attention and graph pooling |
| title_short | CsAGP: Detecting Alzheimer's disease from multimodal images via dual-transformer with cross-attention and graph pooling |
| title_sort | csagp detecting alzheimer s disease from multimodal images via dual transformer with cross attention and graph pooling |
| topic | Alzheimer's disease Vision transformer Multimodal image fusion Deep learning |
| url | http://www.sciencedirect.com/science/article/pii/S1319157823001726 |
| work_keys_str_mv | AT chaoshengtang csagpdetectingalzheimersdiseasefrommultimodalimagesviadualtransformerwithcrossattentionandgraphpooling AT mingyangwei csagpdetectingalzheimersdiseasefrommultimodalimagesviadualtransformerwithcrossattentionandgraphpooling AT jundingsun csagpdetectingalzheimersdiseasefrommultimodalimagesviadualtransformerwithcrossattentionandgraphpooling AT shuihuawang csagpdetectingalzheimersdiseasefrommultimodalimagesviadualtransformerwithcrossattentionandgraphpooling AT yudongzhang csagpdetectingalzheimersdiseasefrommultimodalimagesviadualtransformerwithcrossattentionandgraphpooling |