Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy
Predictive coding provides a computational paradigm for modeling perceptual processing as the construction of representations accounting for causes of sensory inputs. Here, we developed a scalable, deep network architecture for predictive coding that is trained using a gated Hebbian learning rule an...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2021-07-01
|
| Series: | Frontiers in Computational Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fncom.2021.666131/full |
| _version_ | 1818454276949671936 |
|---|---|
| author | Shirin Dora Shirin Dora Sander M. Bohte Sander M. Bohte Cyriel M. A. Pennartz |
| author_facet | Shirin Dora Shirin Dora Sander M. Bohte Sander M. Bohte Cyriel M. A. Pennartz |
| author_sort | Shirin Dora |
| collection | DOAJ |
| description | Predictive coding provides a computational paradigm for modeling perceptual processing as the construction of representations accounting for causes of sensory inputs. Here, we developed a scalable, deep network architecture for predictive coding that is trained using a gated Hebbian learning rule and mimics the feedforward and feedback connectivity of the cortex. After training on image datasets, the models formed latent representations in higher areas that allowed reconstruction of the original images. We analyzed low- and high-level properties such as orientation selectivity, object selectivity and sparseness of neuronal populations in the model. As reported experimentally, image selectivity increased systematically across ascending areas in the model hierarchy. Depending on the strength of regularization factors, sparseness also increased from lower to higher areas. The results suggest a rationale as to why experimental results on sparseness across the cortical hierarchy have been inconsistent. Finally, representations for different object classes became more distinguishable from lower to higher areas. Thus, deep neural networks trained using a gated Hebbian formulation of predictive coding can reproduce several properties associated with neuronal responses along the visual cortical hierarchy. |
| first_indexed | 2024-12-14T21:52:19Z |
| format | Article |
| id | doaj.art-3749bdc028f148818051bdd8867144e3 |
| institution | Directory Open Access Journal |
| issn | 1662-5188 |
| language | English |
| last_indexed | 2024-12-14T21:52:19Z |
| publishDate | 2021-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computational Neuroscience |
| spelling | doaj.art-3749bdc028f148818051bdd8867144e32022-12-21T22:46:13ZengFrontiers Media S.A.Frontiers in Computational Neuroscience1662-51882021-07-011510.3389/fncom.2021.666131666131Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical HierarchyShirin Dora0Shirin Dora1Sander M. Bohte2Sander M. Bohte3Cyriel M. A. Pennartz4Cognitive and Systems Neuroscience Group, Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, NetherlandsIntelligent Systems Research Centre, Ulster University, Londonderry, United KingdomCognitive and Systems Neuroscience Group, Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, NetherlandsMachine Learning Group, Centre of Mathematics and Computer Science, Amsterdam, NetherlandsCognitive and Systems Neuroscience Group, Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, NetherlandsPredictive coding provides a computational paradigm for modeling perceptual processing as the construction of representations accounting for causes of sensory inputs. Here, we developed a scalable, deep network architecture for predictive coding that is trained using a gated Hebbian learning rule and mimics the feedforward and feedback connectivity of the cortex. After training on image datasets, the models formed latent representations in higher areas that allowed reconstruction of the original images. We analyzed low- and high-level properties such as orientation selectivity, object selectivity and sparseness of neuronal populations in the model. As reported experimentally, image selectivity increased systematically across ascending areas in the model hierarchy. Depending on the strength of regularization factors, sparseness also increased from lower to higher areas. The results suggest a rationale as to why experimental results on sparseness across the cortical hierarchy have been inconsistent. Finally, representations for different object classes became more distinguishable from lower to higher areas. Thus, deep neural networks trained using a gated Hebbian formulation of predictive coding can reproduce several properties associated with neuronal responses along the visual cortical hierarchy.https://www.frontiersin.org/articles/10.3389/fncom.2021.666131/fullvisual processingpredictive codingdeep biologically plausible learningselectivitysparsenesssensory neocortex |
| spellingShingle | Shirin Dora Shirin Dora Sander M. Bohte Sander M. Bohte Cyriel M. A. Pennartz Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy Frontiers in Computational Neuroscience visual processing predictive coding deep biologically plausible learning selectivity sparseness sensory neocortex |
| title | Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy |
| title_full | Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy |
| title_fullStr | Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy |
| title_full_unstemmed | Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy |
| title_short | Deep Gated Hebbian Predictive Coding Accounts for Emergence of Complex Neural Response Properties Along the Visual Cortical Hierarchy |
| title_sort | deep gated hebbian predictive coding accounts for emergence of complex neural response properties along the visual cortical hierarchy |
| topic | visual processing predictive coding deep biologically plausible learning selectivity sparseness sensory neocortex |
| url | https://www.frontiersin.org/articles/10.3389/fncom.2021.666131/full |
| work_keys_str_mv | AT shirindora deepgatedhebbianpredictivecodingaccountsforemergenceofcomplexneuralresponsepropertiesalongthevisualcorticalhierarchy AT shirindora deepgatedhebbianpredictivecodingaccountsforemergenceofcomplexneuralresponsepropertiesalongthevisualcorticalhierarchy AT sandermbohte deepgatedhebbianpredictivecodingaccountsforemergenceofcomplexneuralresponsepropertiesalongthevisualcorticalhierarchy AT sandermbohte deepgatedhebbianpredictivecodingaccountsforemergenceofcomplexneuralresponsepropertiesalongthevisualcorticalhierarchy AT cyrielmapennartz deepgatedhebbianpredictivecodingaccountsforemergenceofcomplexneuralresponsepropertiesalongthevisualcorticalhierarchy |