Exploring fMRI RDMs: enhancing model robustness through neurobiological data
Artificial neural networks (ANNs) are sensitive to perturbations and adversarial attacks. One hypothesized solution to adversarial robustness is to align manifolds in the embedded space of neural networks with biologically grounded manifolds. Recent state-of-the-art works that emphasize learning rob...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-12-01
|
| Series: | Frontiers in Computer Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fcomp.2023.1275026/full |
| _version_ | 1797386627415277568 |
|---|---|
| author | William Pickard Kelsey Sikes Huma Jamil Nicholas Chaffee Nathaniel Blanchard Michael Kirby Michael Kirby Chris Peterson |
| author_facet | William Pickard Kelsey Sikes Huma Jamil Nicholas Chaffee Nathaniel Blanchard Michael Kirby Michael Kirby Chris Peterson |
| author_sort | William Pickard |
| collection | DOAJ |
| description | Artificial neural networks (ANNs) are sensitive to perturbations and adversarial attacks. One hypothesized solution to adversarial robustness is to align manifolds in the embedded space of neural networks with biologically grounded manifolds. Recent state-of-the-art works that emphasize learning robust neural representations, rather than optimizing for a specific target task like classification, support the idea that researchers should investigate this hypothesis. While works have shown that fine-tuning ANNs to coincide with biological vision does increase robustness to both perturbations and adversarial attacks, these works have relied on proprietary datasets—the lack of publicly available biological benchmarks makes it difficult to evaluate the efficacy of these claims. Here, we deliver a curated dataset consisting of biological representations of images taken from two commonly used computer vision datasets, ImageNet and COCO, that can be easily integrated into model training and evaluation. Specifically, we take a large functional magnetic resonance imaging (fMRI) dataset (BOLD5000), preprocess it into representational dissimilarity matrices (RDMs), and establish an infrastructure that anyone can use to train models with biologically grounded representations. Using this infrastructure, we investigate the representations of several popular neural networks and find that as networks have been optimized for tasks, their correspondence with biological fidelity has decreased. Additionally, we use a previously unexplored graph-based technique, Fiedler partitioning, to showcase the viability of the biological data, and the potential to extend these analyses by extending RDMs into Laplacian matrices. Overall, our findings demonstrate the potential of utilizing our new biological benchmark to effectively enhance the robustness of models. |
| first_indexed | 2024-03-08T22:12:11Z |
| format | Article |
| id | doaj.art-3fad801f4afb40c282d0a0e5ad15cef2 |
| institution | Directory Open Access Journal |
| issn | 2624-9898 |
| language | English |
| last_indexed | 2024-03-08T22:12:11Z |
| publishDate | 2023-12-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computer Science |
| spelling | doaj.art-3fad801f4afb40c282d0a0e5ad15cef22023-12-19T05:47:11ZengFrontiers Media S.A.Frontiers in Computer Science2624-98982023-12-01510.3389/fcomp.2023.12750261275026Exploring fMRI RDMs: enhancing model robustness through neurobiological dataWilliam Pickard0Kelsey Sikes1Huma Jamil2Nicholas Chaffee3Nathaniel Blanchard4Michael Kirby5Michael Kirby6Chris Peterson7Department of Computer Science, Colorado State University, Fort Collins, CO, United StatesDepartment of Computer Science, Colorado State University, Fort Collins, CO, United StatesDepartment of Computer Science, Colorado State University, Fort Collins, CO, United StatesDepartment of Computer Science, Colorado State University, Fort Collins, CO, United StatesDepartment of Computer Science, Colorado State University, Fort Collins, CO, United StatesDepartment of Computer Science, Colorado State University, Fort Collins, CO, United StatesDepartment of Mathematics, Colorado State University, Fort Collins, CO, United StatesDepartment of Mathematics, Colorado State University, Fort Collins, CO, United StatesArtificial neural networks (ANNs) are sensitive to perturbations and adversarial attacks. One hypothesized solution to adversarial robustness is to align manifolds in the embedded space of neural networks with biologically grounded manifolds. Recent state-of-the-art works that emphasize learning robust neural representations, rather than optimizing for a specific target task like classification, support the idea that researchers should investigate this hypothesis. While works have shown that fine-tuning ANNs to coincide with biological vision does increase robustness to both perturbations and adversarial attacks, these works have relied on proprietary datasets—the lack of publicly available biological benchmarks makes it difficult to evaluate the efficacy of these claims. Here, we deliver a curated dataset consisting of biological representations of images taken from two commonly used computer vision datasets, ImageNet and COCO, that can be easily integrated into model training and evaluation. Specifically, we take a large functional magnetic resonance imaging (fMRI) dataset (BOLD5000), preprocess it into representational dissimilarity matrices (RDMs), and establish an infrastructure that anyone can use to train models with biologically grounded representations. Using this infrastructure, we investigate the representations of several popular neural networks and find that as networks have been optimized for tasks, their correspondence with biological fidelity has decreased. Additionally, we use a previously unexplored graph-based technique, Fiedler partitioning, to showcase the viability of the biological data, and the potential to extend these analyses by extending RDMs into Laplacian matrices. Overall, our findings demonstrate the potential of utilizing our new biological benchmark to effectively enhance the robustness of models.https://www.frontiersin.org/articles/10.3389/fcomp.2023.1275026/fullbrain-inspired neural networkscomputational neurosciencedeep learninggeometric analysisobject recognitionfunctional MRI |
| spellingShingle | William Pickard Kelsey Sikes Huma Jamil Nicholas Chaffee Nathaniel Blanchard Michael Kirby Michael Kirby Chris Peterson Exploring fMRI RDMs: enhancing model robustness through neurobiological data Frontiers in Computer Science brain-inspired neural networks computational neuroscience deep learning geometric analysis object recognition functional MRI |
| title | Exploring fMRI RDMs: enhancing model robustness through neurobiological data |
| title_full | Exploring fMRI RDMs: enhancing model robustness through neurobiological data |
| title_fullStr | Exploring fMRI RDMs: enhancing model robustness through neurobiological data |
| title_full_unstemmed | Exploring fMRI RDMs: enhancing model robustness through neurobiological data |
| title_short | Exploring fMRI RDMs: enhancing model robustness through neurobiological data |
| title_sort | exploring fmri rdms enhancing model robustness through neurobiological data |
| topic | brain-inspired neural networks computational neuroscience deep learning geometric analysis object recognition functional MRI |
| url | https://www.frontiersin.org/articles/10.3389/fcomp.2023.1275026/full |
| work_keys_str_mv | AT williampickard exploringfmrirdmsenhancingmodelrobustnessthroughneurobiologicaldata AT kelseysikes exploringfmrirdmsenhancingmodelrobustnessthroughneurobiologicaldata AT humajamil exploringfmrirdmsenhancingmodelrobustnessthroughneurobiologicaldata AT nicholaschaffee exploringfmrirdmsenhancingmodelrobustnessthroughneurobiologicaldata AT nathanielblanchard exploringfmrirdmsenhancingmodelrobustnessthroughneurobiologicaldata AT michaelkirby exploringfmrirdmsenhancingmodelrobustnessthroughneurobiologicaldata AT michaelkirby exploringfmrirdmsenhancingmodelrobustnessthroughneurobiologicaldata AT chrispeterson exploringfmrirdmsenhancingmodelrobustnessthroughneurobiologicaldata |