Alzheimer’s Disease Prediction via Brain Structural-Functional Deep Fusing Network
Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer’s disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this work, a novel model ter...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10320341/ |
| _version_ | 1797473550078050304 |
|---|---|
| author | Qiankun Zuo Yanyan Shen Ning Zhong C. L. Philip Chen Baiying Lei Shuqiang Wang |
| author_facet | Qiankun Zuo Yanyan Shen Ning Zhong C. L. Philip Chen Baiying Lei Shuqiang Wang |
| author_sort | Qiankun Zuo |
| collection | DOAJ |
| description | Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer’s disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this work, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights into detecting AD-related abnormal neural circuits. |
| first_indexed | 2024-03-09T20:17:03Z |
| format | Article |
| id | doaj.art-faec02987fa74b43993418aa8100200d |
| institution | Directory Open Access Journal |
| issn | 1558-0210 |
| language | English |
| last_indexed | 2024-03-09T20:17:03Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj.art-faec02987fa74b43993418aa8100200d2023-11-24T00:00:16ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1558-02102023-01-01314601461210.1109/TNSRE.2023.333395210320341Alzheimer’s Disease Prediction via Brain Structural-Functional Deep Fusing NetworkQiankun Zuo0Yanyan Shen1https://orcid.org/0000-0003-0639-2925Ning Zhong2C. L. Philip Chen3Baiying Lei4https://orcid.org/0000-0002-3087-2550Shuqiang Wang5https://orcid.org/0000-0003-1119-320XShenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, ChinaShenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, ChinaFaculty of Engineering, Maebashi Institute of Technology, Maebashi, JapanSchool of Computer Science and Engineering, South China University of Technology, Guangzhou, ChinaGuangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, ChinaShenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, ChinaFusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer’s disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this work, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights into detecting AD-related abnormal neural circuits.https://ieeexplore.ieee.org/document/10320341/Multimodal fusionbrain network computingswapping bi-attention mechanismgenerative adversarial strategy |
| spellingShingle | Qiankun Zuo Yanyan Shen Ning Zhong C. L. Philip Chen Baiying Lei Shuqiang Wang Alzheimer’s Disease Prediction via Brain Structural-Functional Deep Fusing Network IEEE Transactions on Neural Systems and Rehabilitation Engineering Multimodal fusion brain network computing swapping bi-attention mechanism generative adversarial strategy |
| title | Alzheimer’s Disease Prediction via Brain Structural-Functional Deep Fusing Network |
| title_full | Alzheimer’s Disease Prediction via Brain Structural-Functional Deep Fusing Network |
| title_fullStr | Alzheimer’s Disease Prediction via Brain Structural-Functional Deep Fusing Network |
| title_full_unstemmed | Alzheimer’s Disease Prediction via Brain Structural-Functional Deep Fusing Network |
| title_short | Alzheimer’s Disease Prediction via Brain Structural-Functional Deep Fusing Network |
| title_sort | alzheimer x2019 s disease prediction via brain structural functional deep fusing network |
| topic | Multimodal fusion brain network computing swapping bi-attention mechanism generative adversarial strategy |
| url | https://ieeexplore.ieee.org/document/10320341/ |
| work_keys_str_mv | AT qiankunzuo alzheimerx2019sdiseasepredictionviabrainstructuralfunctionaldeepfusingnetwork AT yanyanshen alzheimerx2019sdiseasepredictionviabrainstructuralfunctionaldeepfusingnetwork AT ningzhong alzheimerx2019sdiseasepredictionviabrainstructuralfunctionaldeepfusingnetwork AT clphilipchen alzheimerx2019sdiseasepredictionviabrainstructuralfunctionaldeepfusingnetwork AT baiyinglei alzheimerx2019sdiseasepredictionviabrainstructuralfunctionaldeepfusingnetwork AT shuqiangwang alzheimerx2019sdiseasepredictionviabrainstructuralfunctionaldeepfusingnetwork |