Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes
Recent neuroimaging evidence challenges the classical view that face identity and facial expression are processed by segregated neural pathways, showing that information about identity and expression are encoded within common brain regions. This article tests the hypothesis that integrated represent...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Brain Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3425/13/2/296 |
| _version_ | 1797621968886824960 |
|---|---|
| author | Emily Schwartz Kathryn O’Nell Rebecca Saxe Stefano Anzellotti |
| author_facet | Emily Schwartz Kathryn O’Nell Rebecca Saxe Stefano Anzellotti |
| author_sort | Emily Schwartz |
| collection | DOAJ |
| description | Recent neuroimaging evidence challenges the classical view that face identity and facial expression are processed by segregated neural pathways, showing that information about identity and expression are encoded within common brain regions. This article tests the hypothesis that integrated representations of identity and expression arise spontaneously within deep neural networks. A subset of the CelebA dataset is used to train a deep convolutional neural network (DCNN) to label face identity (chance = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.06</mn><mo>%</mo></mrow></semantics></math></inline-formula>, accuracy = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>26.5</mn><mo>%</mo></mrow></semantics></math></inline-formula>), and the FER2013 dataset is used to train a DCNN to label facial expression (chance = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>14.2</mn><mo>%</mo></mrow></semantics></math></inline-formula>, accuracy = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>63.5</mn><mo>%</mo></mrow></semantics></math></inline-formula>). The identity-trained and expression-trained networks each successfully transfer to labeling both face identity and facial expression on the Karolinska Directed Emotional Faces dataset. This study demonstrates that DCNNs trained to recognize face identity and DCNNs trained to recognize facial expression spontaneously develop representations of facial expression and face identity, respectively. Furthermore, a congruence coefficient analysis reveals that features distinguishing between identities and features distinguishing between expressions become increasingly orthogonal from layer to layer, suggesting that deep neural networks disentangle representational subspaces corresponding to different sources. |
| first_indexed | 2024-03-11T09:03:36Z |
| format | Article |
| id | doaj.art-0607fc14c295447cbb910da8887ba3ae |
| institution | Directory Open Access Journal |
| issn | 2076-3425 |
| language | English |
| last_indexed | 2024-03-11T09:03:36Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Brain Sciences |
| spelling | doaj.art-0607fc14c295447cbb910da8887ba3ae2023-11-16T19:29:01ZengMDPI AGBrain Sciences2076-34252023-02-0113229610.3390/brainsci13020296Challenging the Classical View: Recognition of Identity and Expression as Integrated ProcessesEmily Schwartz0Kathryn O’Nell1Rebecca Saxe2Stefano Anzellotti3Department of Psychology and Neuroscience, Boston College, Boston, MA 02467, USADepartment of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755, USADepartment of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USADepartment of Psychology and Neuroscience, Boston College, Boston, MA 02467, USARecent neuroimaging evidence challenges the classical view that face identity and facial expression are processed by segregated neural pathways, showing that information about identity and expression are encoded within common brain regions. This article tests the hypothesis that integrated representations of identity and expression arise spontaneously within deep neural networks. A subset of the CelebA dataset is used to train a deep convolutional neural network (DCNN) to label face identity (chance = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.06</mn><mo>%</mo></mrow></semantics></math></inline-formula>, accuracy = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>26.5</mn><mo>%</mo></mrow></semantics></math></inline-formula>), and the FER2013 dataset is used to train a DCNN to label facial expression (chance = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>14.2</mn><mo>%</mo></mrow></semantics></math></inline-formula>, accuracy = <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>63.5</mn><mo>%</mo></mrow></semantics></math></inline-formula>). The identity-trained and expression-trained networks each successfully transfer to labeling both face identity and facial expression on the Karolinska Directed Emotional Faces dataset. This study demonstrates that DCNNs trained to recognize face identity and DCNNs trained to recognize facial expression spontaneously develop representations of facial expression and face identity, respectively. Furthermore, a congruence coefficient analysis reveals that features distinguishing between identities and features distinguishing between expressions become increasingly orthogonal from layer to layer, suggesting that deep neural networks disentangle representational subspaces corresponding to different sources.https://www.mdpi.com/2076-3425/13/2/296face identityfacial expressiondeep neural networksface recognitionemotions |
| spellingShingle | Emily Schwartz Kathryn O’Nell Rebecca Saxe Stefano Anzellotti Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes Brain Sciences face identity facial expression deep neural networks face recognition emotions |
| title | Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title_full | Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title_fullStr | Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title_full_unstemmed | Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title_short | Challenging the Classical View: Recognition of Identity and Expression as Integrated Processes |
| title_sort | challenging the classical view recognition of identity and expression as integrated processes |
| topic | face identity facial expression deep neural networks face recognition emotions |
| url | https://www.mdpi.com/2076-3425/13/2/296 |
| work_keys_str_mv | AT emilyschwartz challengingtheclassicalviewrecognitionofidentityandexpressionasintegratedprocesses AT kathrynonell challengingtheclassicalviewrecognitionofidentityandexpressionasintegratedprocesses AT rebeccasaxe challengingtheclassicalviewrecognitionofidentityandexpressionasintegratedprocesses AT stefanoanzellotti challengingtheclassicalviewrecognitionofidentityandexpressionasintegratedprocesses |