Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons
In order to better understand how the brain perceives faces, it is important to know what objective drives learning in the ventral visual stream. To answer this question, we model neural responses to faces in the macaque inferotemporal (IT) cortex with a deep self-supervised generative model, β-VAE,...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Springer Nature
2021
|
| _version_ | 1797077472235225088 |
|---|---|
| author | Higgins, I Chang, L Langston, V Hassabis, D Summerfield, C Tsao, D Botvinick, M |
| author_facet | Higgins, I Chang, L Langston, V Hassabis, D Summerfield, C Tsao, D Botvinick, M |
| author_sort | Higgins, I |
| collection | OXFORD |
| description | In order to better understand how the brain perceives faces, it is important to know what objective drives learning in the ventral visual stream. To answer this question, we model neural responses to faces in the macaque inferotemporal (IT) cortex with a deep self-supervised generative model, β-VAE, which disentangles sensory data into interpretable latent factors, such as gender or age. Our results demonstrate a strong correspondence between the generative factors discovered by β-VAE and those coded by single IT neurons, beyond that found for the baselines, including the handcrafted state-of-the-art model of face perception, the Active Appearance Model, and deep classifiers. Moreover, β-VAE is able to reconstruct novel face images using signals from just a handful of cells. Together our results imply that optimising the disentangling objective leads to representations that closely resemble those in the IT at the single unit level. This points at disentangling as a plausible learning objective for the visual brain. |
| first_indexed | 2024-03-07T00:18:31Z |
| format | Journal article |
| id | oxford-uuid:7bb0019a-9a38-44f8-99e4-ec2ce8a6f4a6 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T00:18:31Z |
| publishDate | 2021 |
| publisher | Springer Nature |
| record_format | dspace |
| spelling | oxford-uuid:7bb0019a-9a38-44f8-99e4-ec2ce8a6f4a62022-03-26T20:52:15ZUnsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neuronsJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:7bb0019a-9a38-44f8-99e4-ec2ce8a6f4a6EnglishSymplectic ElementsSpringer Nature2021Higgins, IChang, LLangston, VHassabis, DSummerfield, CTsao, DBotvinick, MIn order to better understand how the brain perceives faces, it is important to know what objective drives learning in the ventral visual stream. To answer this question, we model neural responses to faces in the macaque inferotemporal (IT) cortex with a deep self-supervised generative model, β-VAE, which disentangles sensory data into interpretable latent factors, such as gender or age. Our results demonstrate a strong correspondence between the generative factors discovered by β-VAE and those coded by single IT neurons, beyond that found for the baselines, including the handcrafted state-of-the-art model of face perception, the Active Appearance Model, and deep classifiers. Moreover, β-VAE is able to reconstruct novel face images using signals from just a handful of cells. Together our results imply that optimising the disentangling objective leads to representations that closely resemble those in the IT at the single unit level. This points at disentangling as a plausible learning objective for the visual brain. |
| spellingShingle | Higgins, I Chang, L Langston, V Hassabis, D Summerfield, C Tsao, D Botvinick, M Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons |
| title | Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons |
| title_full | Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons |
| title_fullStr | Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons |
| title_full_unstemmed | Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons |
| title_short | Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons |
| title_sort | unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons |
| work_keys_str_mv | AT higginsi unsuperviseddeeplearningidentifiessemanticdisentanglementinsingleinferotemporalfacepatchneurons AT changl unsuperviseddeeplearningidentifiessemanticdisentanglementinsingleinferotemporalfacepatchneurons AT langstonv unsuperviseddeeplearningidentifiessemanticdisentanglementinsingleinferotemporalfacepatchneurons AT hassabisd unsuperviseddeeplearningidentifiessemanticdisentanglementinsingleinferotemporalfacepatchneurons AT summerfieldc unsuperviseddeeplearningidentifiessemanticdisentanglementinsingleinferotemporalfacepatchneurons AT tsaod unsuperviseddeeplearningidentifiessemanticdisentanglementinsingleinferotemporalfacepatchneurons AT botvinickm unsuperviseddeeplearningidentifiessemanticdisentanglementinsingleinferotemporalfacepatchneurons |