Task-driven convolutional recurrent models of the visual system
Feed-forward convolutional neural networks (CNNs) are currently state-of-the-art for object classification tasks such as ImageNet. Further, they are quantitatively accurate models of temporally-averaged responses of neurons in the primate brain's visual system. However, biological visual system...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020
|
| Online Access: | https://hdl.handle.net/1721.1/126698 |
| _version_ | 1826199792753049600 |
|---|---|
| author | Kubilius, Jonas Kar, Kohitij DiCarlo, James |
| author2 | McGovern Institute for Brain Research at MIT |
| author_facet | McGovern Institute for Brain Research at MIT Kubilius, Jonas Kar, Kohitij DiCarlo, James |
| author_sort | Kubilius, Jonas |
| collection | MIT |
| description | Feed-forward convolutional neural networks (CNNs) are currently state-of-the-art for object classification tasks such as ImageNet. Further, they are quantitatively accurate models of temporally-averaged responses of neurons in the primate brain's visual system. However, biological visual systems have two ubiquitous architectural features not shared with typical CNNs: local recurrence within cortical areas, and long-range feedback from downstream areas to upstream areas. Here we explored the role of recurrence in improving classification performance. We found that standard forms of recurrence (vanilla RNNs and LSTMs) do not perform well within deep CNNs on the ImageNet task. In contrast, novel cells that incorporated two structural features, bypassing and gating, were able to boost task accuracy substantially. We extended these design principles in an automated search over thousands of model architectures, which identified novel local recurrent cells and long-range feedback connections useful for object recognition. Moreover, these task-optimized ConvRNNs matched the dynamics of neural activity in the primate visual system better than feedforward networks, suggesting a role for the brain's recurrent connections in performing difficult visual behaviors. |
| first_indexed | 2024-09-23T11:25:52Z |
| format | Article |
| id | mit-1721.1/126698 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T11:25:52Z |
| publishDate | 2020 |
| publisher | IEEE |
| record_format | dspace |
| spelling | mit-1721.1/1266982022-10-01T03:35:44Z Task-driven convolutional recurrent models of the visual system Kubilius, Jonas Kar, Kohitij DiCarlo, James McGovern Institute for Brain Research at MIT Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Feed-forward convolutional neural networks (CNNs) are currently state-of-the-art for object classification tasks such as ImageNet. Further, they are quantitatively accurate models of temporally-averaged responses of neurons in the primate brain's visual system. However, biological visual systems have two ubiquitous architectural features not shared with typical CNNs: local recurrence within cortical areas, and long-range feedback from downstream areas to upstream areas. Here we explored the role of recurrence in improving classification performance. We found that standard forms of recurrence (vanilla RNNs and LSTMs) do not perform well within deep CNNs on the ImageNet task. In contrast, novel cells that incorporated two structural features, bypassing and gating, were able to boost task accuracy substantially. We extended these design principles in an automated search over thousands of model architectures, which identified novel local recurrent cells and long-range feedback connections useful for object recognition. Moreover, these task-optimized ConvRNNs matched the dynamics of neural activity in the primate visual system better than feedforward networks, suggesting a role for the brain's recurrent connections in performing difficult visual behaviors. Simons Foundation (Grant 325500/542965) European Union. Horizon 2020 Research and Innovation Programme (Grant 705498) National Institutes of Health (U.S.). ǂb National Eye Institute (Grant R01-EY014970) United States. Office of Naval Research. Multidisciplinary University Research Initiative (Grant MURI-114407) 2020-08-20T13:02:13Z 2020-08-20T13:02:13Z 2018-12 2019-09-30T17:20:02Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/126698 Nayebi, Aran et al. “Task-driven convolutional recurrent models of the visual system.” NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing , vol. 2018, 2018, pp. 5295–5306 © 2018 The Author(s) en http://papers.neurips.cc/paper/7775-task-driven-convolutional-recurrent-models-of-the-visual-system NIPS'18: Proceedings of the 32nd International Conference on Neural Information Processing Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf IEEE Neural Information Processing Systems (NIPS) |
| spellingShingle | Kubilius, Jonas Kar, Kohitij DiCarlo, James Task-driven convolutional recurrent models of the visual system |
| title | Task-driven convolutional recurrent models of the visual system |
| title_full | Task-driven convolutional recurrent models of the visual system |
| title_fullStr | Task-driven convolutional recurrent models of the visual system |
| title_full_unstemmed | Task-driven convolutional recurrent models of the visual system |
| title_short | Task-driven convolutional recurrent models of the visual system |
| title_sort | task driven convolutional recurrent models of the visual system |
| url | https://hdl.handle.net/1721.1/126698 |
| work_keys_str_mv | AT kubiliusjonas taskdrivenconvolutionalrecurrentmodelsofthevisualsystem AT karkohitij taskdrivenconvolutionalrecurrentmodelsofthevisualsystem AT dicarlojames taskdrivenconvolutionalrecurrentmodelsofthevisualsystem |