A Hybrid Deep Learning Method for Early and Late Mild Cognitive Impairment Diagnosis With Incomplete Multimodal Data
Multimodality neuroimages have been widely applied to diagnose mild cognitive impairment (MCI). However, the missing data problem is unavoidable. Most previously developed methods first train a generative adversarial network (GAN) to synthesize missing data and then train a classification network wi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-03-01
|
| Series: | Frontiers in Neuroinformatics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fninf.2022.843566/full |
| _version_ | 1818386165049327616 |
|---|---|
| author | Leiming Jin Kun Zhao Yan Zhao Tongtong Che Shuyu Li Shuyu Li |
| author_facet | Leiming Jin Kun Zhao Yan Zhao Tongtong Che Shuyu Li Shuyu Li |
| author_sort | Leiming Jin |
| collection | DOAJ |
| description | Multimodality neuroimages have been widely applied to diagnose mild cognitive impairment (MCI). However, the missing data problem is unavoidable. Most previously developed methods first train a generative adversarial network (GAN) to synthesize missing data and then train a classification network with the completed data. These methods independently train two networks with no information communication. Thus, the resulting GAN cannot focus on the crucial regions that are helpful for classification. To overcome this issue, we propose a hybrid deep learning method. First, a classification network is pretrained with paired MRI and PET images. Afterward, we use the pretrained classification network to guide a GAN by focusing on the features that are helpful for classification. Finally, we synthesize the missing PET images and use them with real MR images to fine-tune the classification model to make it better adapt to the synthesized images. We evaluate our proposed method on the ADNI dataset, and the results show that our method improves the accuracies obtained on the validation and testing sets by 3.84 and 5.82%, respectively. Moreover, our method increases the accuracies for the validation and testing sets by 7.7 and 9.09%, respectively, when we synthesize the missing PET images via our method. An ablation experiment shows that the last two stages are essential for our method. We also compare our method with other state-of-the-art methods, and our method achieves better classification performance. |
| first_indexed | 2024-12-14T03:49:42Z |
| format | Article |
| id | doaj.art-d03f5508f23742ad80022b6726af1e95 |
| institution | Directory Open Access Journal |
| issn | 1662-5196 |
| language | English |
| last_indexed | 2024-12-14T03:49:42Z |
| publishDate | 2022-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroinformatics |
| spelling | doaj.art-d03f5508f23742ad80022b6726af1e952022-12-21T23:18:15ZengFrontiers Media S.A.Frontiers in Neuroinformatics1662-51962022-03-011610.3389/fninf.2022.843566843566A Hybrid Deep Learning Method for Early and Late Mild Cognitive Impairment Diagnosis With Incomplete Multimodal DataLeiming Jin0Kun Zhao1Yan Zhao2Tongtong Che3Shuyu Li4Shuyu Li5Beijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, ChinaBeijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, ChinaBeijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, ChinaBeijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, ChinaBeijing Advanced Innovation Center for Biomedical Engineering, School of Biological Science and Medical Engineering, Beihang University, Beijing, ChinaState Key Lab of Cognition Neuroscience and Learning, Beijing Normal University, Beijing, ChinaMultimodality neuroimages have been widely applied to diagnose mild cognitive impairment (MCI). However, the missing data problem is unavoidable. Most previously developed methods first train a generative adversarial network (GAN) to synthesize missing data and then train a classification network with the completed data. These methods independently train two networks with no information communication. Thus, the resulting GAN cannot focus on the crucial regions that are helpful for classification. To overcome this issue, we propose a hybrid deep learning method. First, a classification network is pretrained with paired MRI and PET images. Afterward, we use the pretrained classification network to guide a GAN by focusing on the features that are helpful for classification. Finally, we synthesize the missing PET images and use them with real MR images to fine-tune the classification model to make it better adapt to the synthesized images. We evaluate our proposed method on the ADNI dataset, and the results show that our method improves the accuracies obtained on the validation and testing sets by 3.84 and 5.82%, respectively. Moreover, our method increases the accuracies for the validation and testing sets by 7.7 and 9.09%, respectively, when we synthesize the missing PET images via our method. An ablation experiment shows that the last two stages are essential for our method. We also compare our method with other state-of-the-art methods, and our method achieves better classification performance.https://www.frontiersin.org/articles/10.3389/fninf.2022.843566/fullMCIGANclassificationincomplete datamultimodality |
| spellingShingle | Leiming Jin Kun Zhao Yan Zhao Tongtong Che Shuyu Li Shuyu Li A Hybrid Deep Learning Method for Early and Late Mild Cognitive Impairment Diagnosis With Incomplete Multimodal Data Frontiers in Neuroinformatics MCI GAN classification incomplete data multimodality |
| title | A Hybrid Deep Learning Method for Early and Late Mild Cognitive Impairment Diagnosis With Incomplete Multimodal Data |
| title_full | A Hybrid Deep Learning Method for Early and Late Mild Cognitive Impairment Diagnosis With Incomplete Multimodal Data |
| title_fullStr | A Hybrid Deep Learning Method for Early and Late Mild Cognitive Impairment Diagnosis With Incomplete Multimodal Data |
| title_full_unstemmed | A Hybrid Deep Learning Method for Early and Late Mild Cognitive Impairment Diagnosis With Incomplete Multimodal Data |
| title_short | A Hybrid Deep Learning Method for Early and Late Mild Cognitive Impairment Diagnosis With Incomplete Multimodal Data |
| title_sort | hybrid deep learning method for early and late mild cognitive impairment diagnosis with incomplete multimodal data |
| topic | MCI GAN classification incomplete data multimodality |
| url | https://www.frontiersin.org/articles/10.3389/fninf.2022.843566/full |
| work_keys_str_mv | AT leimingjin ahybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT kunzhao ahybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT yanzhao ahybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT tongtongche ahybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT shuyuli ahybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT shuyuli ahybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT leimingjin hybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT kunzhao hybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT yanzhao hybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT tongtongche hybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT shuyuli hybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata AT shuyuli hybriddeeplearningmethodforearlyandlatemildcognitiveimpairmentdiagnosiswithincompletemultimodaldata |