Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes
Recent neuroimaging evidence challenges the classical view that face identity and facial expression are processed by segregated neural pathways, showing that information about identity and expression are encoded within common brain regions. This article tests the hypothesis that integrated represent...
| Κύριοι συγγραφείς: | , , , |
|---|---|
| Άλλοι συγγραφείς: | |
| Μορφή: | Άρθρο |
| Έκδοση: |
Multidisciplinary Digital Publishing Institute
2023
|
| Διαθέσιμο Online: | https://hdl.handle.net/1721.1/148018 |
| _version_ | 1826197244503654400 |
|---|---|
| author | Schwartz, Emily O’Nell, Kathryn Saxe, Rebecca Anzellotti, Stefano |
| author2 | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences |
| author_facet | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Schwartz, Emily O’Nell, Kathryn Saxe, Rebecca Anzellotti, Stefano |
| author_sort | Schwartz, Emily |
| collection | MIT |
| description | Recent neuroimaging evidence challenges the classical view that face identity and facial expression are processed by segregated neural pathways, showing that information about identity and expression are encoded within common brain regions. This article tests the hypothesis that integrated representations of identity and expression arise spontaneously within deep neural networks. A subset of the CelebA dataset is used to train a deep convolutional neural network (DCNN) to label face identity (chance = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.06</mn><mo>%</mo></mrow></semantics></math></inline-formula>, accuracy = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>26.5</mn><mo>%</mo></mrow></semantics></math></inline-formula>), and the FER2013 dataset is used to train a DCNN to label facial expression (chance = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>14.2</mn><mo>%</mo></mrow></semantics></math></inline-formula>, accuracy = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>63.5</mn><mo>%</mo></mrow></semantics></math></inline-formula>). The identity-trained and expression-trained networks each successfully transfer to labeling both face identity and facial expression on the Karolinska Directed Emotional Faces dataset. This study demonstrates that DCNNs trained to recognize face identity and DCNNs trained to recognize facial expression spontaneously develop representations of facial expression and face identity, respectively. Furthermore, a congruence coefficient analysis reveals that features distinguishing between identities and features distinguishing between expressions become increasingly orthogonal from layer to layer, suggesting that deep neural networks disentangle representational subspaces corresponding to different sources. |
| first_indexed | 2024-09-23T10:44:44Z |
| format | Article |
| id | mit-1721.1/148018 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T10:44:44Z |
| publishDate | 2023 |
| publisher | Multidisciplinary Digital Publishing Institute |
| record_format | dspace |
| spelling | mit-1721.1/1480182023-02-11T03:37:04Z Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes Schwartz, Emily O’Nell, Kathryn Saxe, Rebecca Anzellotti, Stefano Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Recent neuroimaging evidence challenges the classical view that face identity and facial expression are processed by segregated neural pathways, showing that information about identity and expression are encoded within common brain regions. This article tests the hypothesis that integrated representations of identity and expression arise spontaneously within deep neural networks. A subset of the CelebA dataset is used to train a deep convolutional neural network (DCNN) to label face identity (chance = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.06</mn><mo>%</mo></mrow></semantics></math></inline-formula>, accuracy = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>26.5</mn><mo>%</mo></mrow></semantics></math></inline-formula>), and the FER2013 dataset is used to train a DCNN to label facial expression (chance = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>14.2</mn><mo>%</mo></mrow></semantics></math></inline-formula>, accuracy = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>63.5</mn><mo>%</mo></mrow></semantics></math></inline-formula>). The identity-trained and expression-trained networks each successfully transfer to labeling both face identity and facial expression on the Karolinska Directed Emotional Faces dataset. This study demonstrates that DCNNs trained to recognize face identity and DCNNs trained to recognize facial expression spontaneously develop representations of facial expression and face identity, respectively. Furthermore, a congruence coefficient analysis reveals that features distinguishing between identities and features distinguishing between expressions become increasingly orthogonal from layer to layer, suggesting that deep neural networks disentangle representational subspaces corresponding to different sources. 2023-02-10T16:17:15Z 2023-02-10T16:17:15Z 2023-02-10 2023-02-10T14:28:43Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/148018 Brain Sciences 13 (2): 296 (2023) PUBLISHER_CC http://dx.doi.org/10.3390/brainsci13020296 Creative Commons Attribution https://creativecommons.org/licenses/by/4.0/ application/pdf Multidisciplinary Digital Publishing Institute Multidisciplinary Digital Publishing Institute |
| spellingShingle | Schwartz, Emily O’Nell, Kathryn Saxe, Rebecca Anzellotti, Stefano Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title | Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title_full | Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title_fullStr | Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title_full_unstemmed | Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title_short | Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title_sort | challenging the classical view recognition of identity and expression as integrated processes |
| url | https://hdl.handle.net/1721.1/148018 |
| work_keys_str_mv | AT schwartzemily challengingtheclassicalviewrecognitionofidentityandexpressionasintegratedprocesses AT onellkathryn challengingtheclassicalviewrecognitionofidentityandexpressionasintegratedprocesses AT saxerebecca challengingtheclassicalviewrecognitionofidentityandexpressionasintegratedprocesses AT anzellottistefano challengingtheclassicalviewrecognitionofidentityandexpressionasintegratedprocesses |