Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence
The complex multi-stage architecture of cortical visual pathways provides the neural basis for efficient visual object recognition in humans. However, the stage-wise computations therein remain poorly understood. Here, we compared temporal (magnetoencephalography) and spatial (functional MRI) visual...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Springer Nature
2016
|
| Online Access: | http://hdl.handle.net/1721.1/103585 https://orcid.org/0000-0002-0007-3352 https://orcid.org/0000-0003-4915-0256 |
| _version_ | 1826209566901141504 |
|---|---|
| author | Cichy, Radoslaw Khosla, Aditya Pantazis, Dimitrios Torralba, Antonio Oliva, Aude |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Cichy, Radoslaw Khosla, Aditya Pantazis, Dimitrios Torralba, Antonio Oliva, Aude |
| author_sort | Cichy, Radoslaw |
| collection | MIT |
| description | The complex multi-stage architecture of cortical visual pathways provides the neural basis for efficient visual object recognition in humans. However, the stage-wise computations therein remain poorly understood. Here, we compared temporal (magnetoencephalography) and spatial (functional MRI) visual brain representations with representations in an artificial deep neural network (DNN) tuned to the statistics of real-world visual recognition. We showed that the DNN captured the stages of human visual processing in both time and space from early visual areas towards the dorsal and ventral streams. Further investigation of crucial DNN parameters revealed that while model architecture was important, training on real-world categorization was necessary to enforce spatio-temporal hierarchical relationships with the brain. Together our results provide an algorithmically informed view on the spatio-temporal dynamics of visual object recognition in the human visual brain. |
| first_indexed | 2024-09-23T14:24:31Z |
| format | Article |
| id | mit-1721.1/103585 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T14:24:31Z |
| publishDate | 2016 |
| publisher | Springer Nature |
| record_format | dspace |
| spelling | mit-1721.1/1035852022-10-01T21:11:14Z Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence Cichy, Radoslaw Khosla, Aditya Pantazis, Dimitrios Torralba, Antonio Oliva, Aude Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science McGovern Institute for Brain Research at MIT Cichy, Radoslaw Khosla, Aditya Pantazis, Dimitrios Torralba, Antonio Oliva, Aude The complex multi-stage architecture of cortical visual pathways provides the neural basis for efficient visual object recognition in humans. However, the stage-wise computations therein remain poorly understood. Here, we compared temporal (magnetoencephalography) and spatial (functional MRI) visual brain representations with representations in an artificial deep neural network (DNN) tuned to the statistics of real-world visual recognition. We showed that the DNN captured the stages of human visual processing in both time and space from early visual areas towards the dorsal and ventral streams. Further investigation of crucial DNN parameters revealed that while model architecture was important, training on real-world categorization was necessary to enforce spatio-temporal hierarchical relationships with the brain. Together our results provide an algorithmically informed view on the spatio-temporal dynamics of visual object recognition in the human visual brain. National Eye Institute (EY020484) Google (Firm) (Google Research Faculty Award) Alexander von Humboldt-Stiftung (Feodor Lynen Postdoctoral Fellowship) Deutsche Forschungsgemeinschaft (Emmy Noether Program, CI 241/1-1) McGovern Institute Neurotechnology (MINT) program National Science Foundation (U.S.) (NSF Award 1532591) 2016-07-13T15:28:55Z 2016-07-13T15:28:55Z 2016-06 2016-01 Article http://purl.org/eprint/type/JournalArticle 2045-2322 http://hdl.handle.net/1721.1/103585 Cichy, Radoslaw Martin, Aditya Khosla, Dimitrios Pantazis, Antonio Torralba, and Aude Oliva. "Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence." Scientific Reports 6, Article number:27755 (2016), p.1-12. https://orcid.org/0000-0002-0007-3352 https://orcid.org/0000-0003-4915-0256 en_US http://dx.doi.org/10.1038/srep27755 Scientific Reports Creative Commons Attribution 4.0 International License http://creativecommons.org/licenses/by/4.0/ application/pdf Springer Nature Scientific Reports |
| spellingShingle | Cichy, Radoslaw Khosla, Aditya Pantazis, Dimitrios Torralba, Antonio Oliva, Aude Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence |
| title | Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence |
| title_full | Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence |
| title_fullStr | Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence |
| title_full_unstemmed | Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence |
| title_short | Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence |
| title_sort | comparison of deep neural networks to spatio temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence |
| url | http://hdl.handle.net/1721.1/103585 https://orcid.org/0000-0002-0007-3352 https://orcid.org/0000-0003-4915-0256 |
| work_keys_str_mv | AT cichyradoslaw comparisonofdeepneuralnetworkstospatiotemporalcorticaldynamicsofhumanvisualobjectrecognitionrevealshierarchicalcorrespondence AT khoslaaditya comparisonofdeepneuralnetworkstospatiotemporalcorticaldynamicsofhumanvisualobjectrecognitionrevealshierarchicalcorrespondence AT pantazisdimitrios comparisonofdeepneuralnetworkstospatiotemporalcorticaldynamicsofhumanvisualobjectrecognitionrevealshierarchicalcorrespondence AT torralbaantonio comparisonofdeepneuralnetworkstospatiotemporalcorticaldynamicsofhumanvisualobjectrecognitionrevealshierarchicalcorrespondence AT olivaaude comparisonofdeepneuralnetworkstospatiotemporalcorticaldynamicsofhumanvisualobjectrecognitionrevealshierarchicalcorrespondence |