Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder
Recent medical imaging technologies, specifically functional magnetic resonance imaging (fMRI), have advanced the diagnosis of neurological and neurodevelopmental disorders by allowing scientists and physicians to observe the activity within and between different regions of the brain. Deep learning...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-06-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/6/6/47 |
| _version_ | 1797565641327116288 |
|---|---|
| author | Michelle Tang Pulkit Kumar Hao Chen Abhinav Shrivastava |
| author_facet | Michelle Tang Pulkit Kumar Hao Chen Abhinav Shrivastava |
| author_sort | Michelle Tang |
| collection | DOAJ |
| description | Recent medical imaging technologies, specifically functional magnetic resonance imaging (fMRI), have advanced the diagnosis of neurological and neurodevelopmental disorders by allowing scientists and physicians to observe the activity within and between different regions of the brain. Deep learning methods have frequently been implemented to analyze images produced by such technologies and perform disease classification tasks; however, current state-of-the-art approaches do not take advantage of all the information offered by fMRI scans. In this paper, we propose a deep multimodal model that learns a joint representation from two types of connectomic data offered by fMRI scans. Incorporating two functional imaging modalities in an automated end-to-end autism diagnosis system will offer a more comprehensive picture of the neural activity, and thus allow for more accurate diagnoses. Our multimodal training strategy achieves a classification accuracy of 74% and a recall of 95%, as well as an F1 score of 0.805, and its overall performance is superior to using only one type of functional data. |
| first_indexed | 2024-03-10T19:16:00Z |
| format | Article |
| id | doaj.art-5ebf43b1e1bd43aeba2db75763e43927 |
| institution | Directory Open Access Journal |
| issn | 2313-433X |
| language | English |
| last_indexed | 2024-03-10T19:16:00Z |
| publishDate | 2020-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj.art-5ebf43b1e1bd43aeba2db75763e439272023-11-20T03:24:39ZengMDPI AGJournal of Imaging2313-433X2020-06-01664710.3390/jimaging6060047Deep Multimodal Learning for the Diagnosis of Autism Spectrum DisorderMichelle Tang0Pulkit Kumar1Hao Chen2Abhinav Shrivastava3Science, Math, and Computer Science Magnet Program, Montgomery Blair High School, Silver Spring, MD 20901, USAInstitute for Advanced Computer Studies, University of Maryland College Park, College Park, MD 20742, USAInstitute for Advanced Computer Studies, University of Maryland College Park, College Park, MD 20742, USAInstitute for Advanced Computer Studies, University of Maryland College Park, College Park, MD 20742, USARecent medical imaging technologies, specifically functional magnetic resonance imaging (fMRI), have advanced the diagnosis of neurological and neurodevelopmental disorders by allowing scientists and physicians to observe the activity within and between different regions of the brain. Deep learning methods have frequently been implemented to analyze images produced by such technologies and perform disease classification tasks; however, current state-of-the-art approaches do not take advantage of all the information offered by fMRI scans. In this paper, we propose a deep multimodal model that learns a joint representation from two types of connectomic data offered by fMRI scans. Incorporating two functional imaging modalities in an automated end-to-end autism diagnosis system will offer a more comprehensive picture of the neural activity, and thus allow for more accurate diagnoses. Our multimodal training strategy achieves a classification accuracy of 74% and a recall of 95%, as well as an F1 score of 0.805, and its overall performance is superior to using only one type of functional data.https://www.mdpi.com/2313-433X/6/6/47deep learningmultimodal learningconvolutional neural networksautismfMRI |
| spellingShingle | Michelle Tang Pulkit Kumar Hao Chen Abhinav Shrivastava Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder Journal of Imaging deep learning multimodal learning convolutional neural networks autism fMRI |
| title | Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder |
| title_full | Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder |
| title_fullStr | Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder |
| title_full_unstemmed | Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder |
| title_short | Deep Multimodal Learning for the Diagnosis of Autism Spectrum Disorder |
| title_sort | deep multimodal learning for the diagnosis of autism spectrum disorder |
| topic | deep learning multimodal learning convolutional neural networks autism fMRI |
| url | https://www.mdpi.com/2313-433X/6/6/47 |
| work_keys_str_mv | AT michelletang deepmultimodallearningforthediagnosisofautismspectrumdisorder AT pulkitkumar deepmultimodallearningforthediagnosisofautismspectrumdisorder AT haochen deepmultimodallearningforthediagnosisofautismspectrumdisorder AT abhinavshrivastava deepmultimodallearningforthediagnosisofautismspectrumdisorder |