Performance-optimized hierarchical models predict neural responses in higher visual cortex
The ventral visual stream underlies key human visual object recognition abilities. However, neural encoding in the higher areas of the ventral stream remains poorly understood. Here, we describe a modeling approach that yields a quantitatively accurate model of inferior temporal (IT) cortex, the hig...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
National Academy of Sciences (U.S.)
2015
|
| Online Access: | http://hdl.handle.net/1721.1/92787 https://orcid.org/0000-0002-6003-3280 https://orcid.org/0000-0002-1592-5896 https://orcid.org/0000-0001-7779-2219 |
| _version_ | 1826210128538370048 |
|---|---|
| author | Yamins, Daniel L. K. Hong, Ha Cadieu, Charles Solomon, Ethan A. Seibert, Darren Allen DiCarlo, James |
| author2 | Massachusetts Institute of Technology. Institute for Medical Engineering & Science |
| author_facet | Massachusetts Institute of Technology. Institute for Medical Engineering & Science Yamins, Daniel L. K. Hong, Ha Cadieu, Charles Solomon, Ethan A. Seibert, Darren Allen DiCarlo, James |
| author_sort | Yamins, Daniel L. K. |
| collection | MIT |
| description | The ventral visual stream underlies key human visual object recognition abilities. However, neural encoding in the higher areas of the ventral stream remains poorly understood. Here, we describe a modeling approach that yields a quantitatively accurate model of inferior temporal (IT) cortex, the highest ventral cortical area. Using high-throughput computational techniques, we discovered that, within a class of biologically plausible hierarchical neural network models, there is a strong correlation between a model’s categorization performance and its ability to predict individual IT neural unit response data. To pursue this idea, we then identified a high-performing neural network that matches human performance on a range of recognition tasks. Critically, even though we did not constrain this model to match neural data, its top output layer turns out to be highly predictive of IT spiking responses to complex naturalistic images at both the single site and population levels. Moreover, the model’s intermediate layers are highly predictive of neural responses in the V4 cortex, a midlevel visual area that provides the dominant cortical input to IT. These results show that performance optimization—applied in a biologically appropriate model class—can be used to build quantitative predictive models of neural processing. |
| first_indexed | 2024-09-23T14:44:20Z |
| format | Article |
| id | mit-1721.1/92787 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T14:44:20Z |
| publishDate | 2015 |
| publisher | National Academy of Sciences (U.S.) |
| record_format | dspace |
| spelling | mit-1721.1/927872022-09-29T10:14:17Z Performance-optimized hierarchical models predict neural responses in higher visual cortex Yamins, Daniel L. K. Hong, Ha Cadieu, Charles Solomon, Ethan A. Seibert, Darren Allen DiCarlo, James Massachusetts Institute of Technology. Institute for Medical Engineering & Science Harvard University--MIT Division of Health Sciences and Technology Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences McGovern Institute for Brain Research at MIT Yamins, Daniel L. K. Hong, Ha Cadieu, Charles Solomon, Ethan A. Seibert, Darren Allen DiCarlo, James The ventral visual stream underlies key human visual object recognition abilities. However, neural encoding in the higher areas of the ventral stream remains poorly understood. Here, we describe a modeling approach that yields a quantitatively accurate model of inferior temporal (IT) cortex, the highest ventral cortical area. Using high-throughput computational techniques, we discovered that, within a class of biologically plausible hierarchical neural network models, there is a strong correlation between a model’s categorization performance and its ability to predict individual IT neural unit response data. To pursue this idea, we then identified a high-performing neural network that matches human performance on a range of recognition tasks. Critically, even though we did not constrain this model to match neural data, its top output layer turns out to be highly predictive of IT spiking responses to complex naturalistic images at both the single site and population levels. Moreover, the model’s intermediate layers are highly predictive of neural responses in the V4 cortex, a midlevel visual area that provides the dominant cortical input to IT. These results show that performance optimization—applied in a biologically appropriate model class—can be used to build quantitative predictive models of neural processing. National Science Foundation (U.S.) (Grant IS 0964269) National Eye Institute (Grant R01-EY014970) 2015-01-12T16:36:11Z 2015-01-12T16:36:11Z 2014-05 2014-03 Article http://purl.org/eprint/type/JournalArticle 0027-8424 1091-6490 http://hdl.handle.net/1721.1/92787 Yamins, Daniel L. K., Ha Hong, Charles F. Cadieu, Ethan A. Solomon, Darren Seibert, and James J. DiCarlo. “Performance-Optimized Hierarchical Models Predict Neural Responses in Higher Visual Cortex.” Proceedings of the National Academy of Sciences 111, no. 23 (May 8, 2014): 8619–8624. https://orcid.org/0000-0002-6003-3280 https://orcid.org/0000-0002-1592-5896 https://orcid.org/0000-0001-7779-2219 en_US http://dx.doi.org/10.1073/pnas.1403112111 Proceedings of the National Academy of Sciences of the United States of America Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf National Academy of Sciences (U.S.) PNAS |
| spellingShingle | Yamins, Daniel L. K. Hong, Ha Cadieu, Charles Solomon, Ethan A. Seibert, Darren Allen DiCarlo, James Performance-optimized hierarchical models predict neural responses in higher visual cortex |
| title | Performance-optimized hierarchical models predict neural responses in higher visual cortex |
| title_full | Performance-optimized hierarchical models predict neural responses in higher visual cortex |
| title_fullStr | Performance-optimized hierarchical models predict neural responses in higher visual cortex |
| title_full_unstemmed | Performance-optimized hierarchical models predict neural responses in higher visual cortex |
| title_short | Performance-optimized hierarchical models predict neural responses in higher visual cortex |
| title_sort | performance optimized hierarchical models predict neural responses in higher visual cortex |
| url | http://hdl.handle.net/1721.1/92787 https://orcid.org/0000-0002-6003-3280 https://orcid.org/0000-0002-1592-5896 https://orcid.org/0000-0001-7779-2219 |
| work_keys_str_mv | AT yaminsdaniellk performanceoptimizedhierarchicalmodelspredictneuralresponsesinhighervisualcortex AT hongha performanceoptimizedhierarchicalmodelspredictneuralresponsesinhighervisualcortex AT cadieucharles performanceoptimizedhierarchicalmodelspredictneuralresponsesinhighervisualcortex AT solomonethana performanceoptimizedhierarchicalmodelspredictneuralresponsesinhighervisualcortex AT seibertdarrenallen performanceoptimizedhierarchicalmodelspredictneuralresponsesinhighervisualcortex AT dicarlojames performanceoptimizedhierarchicalmodelspredictneuralresponsesinhighervisualcortex |