Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders
Abstract Background: The integration of the information encoded in multiparametric MRI images can enhance the performance of machine-learning classifiers. In this study, we investigate whether the combination of structural and functional MRI might improve the performances of a deep learning (DL) mod...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2024-01-01
|
| Series: | Brain Informatics |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40708-023-00217-4 |
| _version_ | 1797355447070490624 |
|---|---|
| author | Sara Saponaro Francesca Lizzi Giacomo Serra Francesca Mainas Piernicola Oliva Alessia Giuliano Sara Calderoni Alessandra Retico |
| author_facet | Sara Saponaro Francesca Lizzi Giacomo Serra Francesca Mainas Piernicola Oliva Alessia Giuliano Sara Calderoni Alessandra Retico |
| author_sort | Sara Saponaro |
| collection | DOAJ |
| description | Abstract Background: The integration of the information encoded in multiparametric MRI images can enhance the performance of machine-learning classifiers. In this study, we investigate whether the combination of structural and functional MRI might improve the performances of a deep learning (DL) model trained to discriminate subjects with Autism Spectrum Disorders (ASD) with respect to typically developing controls (TD). Material and methods We analyzed both structural and functional MRI brain scans publicly available within the ABIDE I and II data collections. We considered 1383 male subjects with age between 5 and 40 years, including 680 subjects with ASD and 703 TD from 35 different acquisition sites. We extracted morphometric and functional brain features from MRI scans with the Freesurfer and the CPAC analysis packages, respectively. Then, due to the multisite nature of the dataset, we implemented a data harmonization protocol. The ASD vs. TD classification was carried out with a multiple-input DL model, consisting in a neural network which generates a fixed-length feature representation of the data of each modality (FR-NN), and a Dense Neural Network for classification (C-NN). Specifically, we implemented a joint fusion approach to multiple source data integration. The main advantage of the latter is that the loss is propagated back to the FR-NN during the training, thus creating informative feature representations for each data modality. Then, a C-NN, with a number of layers and neurons per layer to be optimized during the model training, performs the ASD-TD discrimination. The performance was evaluated by computing the Area under the Receiver Operating Characteristic curve within a nested 10-fold cross-validation. The brain features that drive the DL classification were identified by the SHAP explainability framework. Results The AUC values of 0.66±0.05 and of 0.76±0.04 were obtained in the ASD vs. TD discrimination when only structural or functional features are considered, respectively. The joint fusion approach led to an AUC of 0.78±0.04. The set of structural and functional connectivity features identified as the most important for the two-class discrimination supports the idea that brain changes tend to occur in individuals with ASD in regions belonging to the Default Mode Network and to the Social Brain. Conclusions Our results demonstrate that the multimodal joint fusion approach outperforms the classification results obtained with data acquired by a single MRI modality as it efficiently exploits the complementarity of structural and functional brain information. |
| first_indexed | 2024-03-08T14:11:22Z |
| format | Article |
| id | doaj.art-95e49b4b3f694885875c848e78650032 |
| institution | Directory Open Access Journal |
| issn | 2198-4018 2198-4026 |
| language | English |
| last_indexed | 2024-03-08T14:11:22Z |
| publishDate | 2024-01-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Brain Informatics |
| spelling | doaj.art-95e49b4b3f694885875c848e786500322024-01-14T12:43:03ZengSpringerOpenBrain Informatics2198-40182198-40262024-01-0111111310.1186/s40708-023-00217-4Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disordersSara Saponaro0Francesca Lizzi1Giacomo Serra2Francesca Mainas3Piernicola Oliva4Alessia Giuliano5Sara Calderoni6Alessandra Retico7Medical Physics School, University of PisaNational Institute for Nuclear Physics (INFN), Pisa DivisionDepartment of Physics, University of CagliariINFN, Cagliari DivisionINFN, Cagliari DivisionUnit of Medical Physics, Pisa University Hospital “Azienda Ospedaliero-Universitaria Pisana”Developmental Psychiatry Unit - IRCCS Stella Maris FoundationNational Institute for Nuclear Physics (INFN), Pisa DivisionAbstract Background: The integration of the information encoded in multiparametric MRI images can enhance the performance of machine-learning classifiers. In this study, we investigate whether the combination of structural and functional MRI might improve the performances of a deep learning (DL) model trained to discriminate subjects with Autism Spectrum Disorders (ASD) with respect to typically developing controls (TD). Material and methods We analyzed both structural and functional MRI brain scans publicly available within the ABIDE I and II data collections. We considered 1383 male subjects with age between 5 and 40 years, including 680 subjects with ASD and 703 TD from 35 different acquisition sites. We extracted morphometric and functional brain features from MRI scans with the Freesurfer and the CPAC analysis packages, respectively. Then, due to the multisite nature of the dataset, we implemented a data harmonization protocol. The ASD vs. TD classification was carried out with a multiple-input DL model, consisting in a neural network which generates a fixed-length feature representation of the data of each modality (FR-NN), and a Dense Neural Network for classification (C-NN). Specifically, we implemented a joint fusion approach to multiple source data integration. The main advantage of the latter is that the loss is propagated back to the FR-NN during the training, thus creating informative feature representations for each data modality. Then, a C-NN, with a number of layers and neurons per layer to be optimized during the model training, performs the ASD-TD discrimination. The performance was evaluated by computing the Area under the Receiver Operating Characteristic curve within a nested 10-fold cross-validation. The brain features that drive the DL classification were identified by the SHAP explainability framework. Results The AUC values of 0.66±0.05 and of 0.76±0.04 were obtained in the ASD vs. TD discrimination when only structural or functional features are considered, respectively. The joint fusion approach led to an AUC of 0.78±0.04. The set of structural and functional connectivity features identified as the most important for the two-class discrimination supports the idea that brain changes tend to occur in individuals with ASD in regions belonging to the Default Mode Network and to the Social Brain. Conclusions Our results demonstrate that the multimodal joint fusion approach outperforms the classification results obtained with data acquired by a single MRI modality as it efficiently exploits the complementarity of structural and functional brain information.https://doi.org/10.1186/s40708-023-00217-4Multi-modal machine learningDeep LearningAutism spectrum disordersABIDEStructural MRIFunctional connectivity |
| spellingShingle | Sara Saponaro Francesca Lizzi Giacomo Serra Francesca Mainas Piernicola Oliva Alessia Giuliano Sara Calderoni Alessandra Retico Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders Brain Informatics Multi-modal machine learning Deep Learning Autism spectrum disorders ABIDE Structural MRI Functional connectivity |
| title | Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders |
| title_full | Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders |
| title_fullStr | Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders |
| title_full_unstemmed | Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders |
| title_short | Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders |
| title_sort | deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders |
| topic | Multi-modal machine learning Deep Learning Autism spectrum disorders ABIDE Structural MRI Functional connectivity |
| url | https://doi.org/10.1186/s40708-023-00217-4 |
| work_keys_str_mv | AT sarasaponaro deeplearningbasedjointfusionapproachtoexploitanatomicalandfunctionalbraininformationinautismspectrumdisorders AT francescalizzi deeplearningbasedjointfusionapproachtoexploitanatomicalandfunctionalbraininformationinautismspectrumdisorders AT giacomoserra deeplearningbasedjointfusionapproachtoexploitanatomicalandfunctionalbraininformationinautismspectrumdisorders AT francescamainas deeplearningbasedjointfusionapproachtoexploitanatomicalandfunctionalbraininformationinautismspectrumdisorders AT piernicolaoliva deeplearningbasedjointfusionapproachtoexploitanatomicalandfunctionalbraininformationinautismspectrumdisorders AT alessiagiuliano deeplearningbasedjointfusionapproachtoexploitanatomicalandfunctionalbraininformationinautismspectrumdisorders AT saracalderoni deeplearningbasedjointfusionapproachtoexploitanatomicalandfunctionalbraininformationinautismspectrumdisorders AT alessandraretico deeplearningbasedjointfusionapproachtoexploitanatomicalandfunctionalbraininformationinautismspectrumdisorders |