Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity.
Facial expressions are fundamental to interpersonal communication, including social interaction, and allow people of different ages, cultures, and languages to quickly and reliably convey emotional information. Historically, facial expression research has followed from discrete emotion theories, whi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2019-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0211735 |
| _version_ | 1818726315355799552 |
|---|---|
| author | Nathaniel Haines Matthew W Southward Jennifer S Cheavens Theodore Beauchaine Woo-Young Ahn |
| author_facet | Nathaniel Haines Matthew W Southward Jennifer S Cheavens Theodore Beauchaine Woo-Young Ahn |
| author_sort | Nathaniel Haines |
| collection | DOAJ |
| description | Facial expressions are fundamental to interpersonal communication, including social interaction, and allow people of different ages, cultures, and languages to quickly and reliably convey emotional information. Historically, facial expression research has followed from discrete emotion theories, which posit a limited number of distinct affective states that are represented with specific patterns of facial action. Much less work has focused on dimensional features of emotion, particularly positive and negative affect intensity. This is likely, in part, because achieving inter-rater reliability for facial action and affect intensity ratings is painstaking and labor-intensive. We use computer-vision and machine learning (CVML) to identify patterns of facial actions in 4,648 video recordings of 125 human participants, which show strong correspondences to positive and negative affect intensity ratings obtained from highly trained coders. Our results show that CVML can both (1) determine the importance of different facial actions that human coders use to derive positive and negative affective ratings when combined with interpretable machine learning methods, and (2) efficiently automate positive and negative affect intensity coding on large facial expression databases. Further, we show that CVML can be applied to individual human judges to infer which facial actions they use to generate perceptual emotion ratings from facial expressions. |
| first_indexed | 2024-12-17T21:56:15Z |
| format | Article |
| id | doaj.art-8fb637d75cf6432ea38ae78a5430ab0d |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-17T21:56:15Z |
| publishDate | 2019-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-8fb637d75cf6432ea38ae78a5430ab0d2022-12-21T21:31:07ZengPublic Library of Science (PLoS)PLoS ONE1932-62032019-01-01142e021173510.1371/journal.pone.0211735Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity.Nathaniel HainesMatthew W SouthwardJennifer S CheavensTheodore BeauchaineWoo-Young AhnFacial expressions are fundamental to interpersonal communication, including social interaction, and allow people of different ages, cultures, and languages to quickly and reliably convey emotional information. Historically, facial expression research has followed from discrete emotion theories, which posit a limited number of distinct affective states that are represented with specific patterns of facial action. Much less work has focused on dimensional features of emotion, particularly positive and negative affect intensity. This is likely, in part, because achieving inter-rater reliability for facial action and affect intensity ratings is painstaking and labor-intensive. We use computer-vision and machine learning (CVML) to identify patterns of facial actions in 4,648 video recordings of 125 human participants, which show strong correspondences to positive and negative affect intensity ratings obtained from highly trained coders. Our results show that CVML can both (1) determine the importance of different facial actions that human coders use to derive positive and negative affective ratings when combined with interpretable machine learning methods, and (2) efficiently automate positive and negative affect intensity coding on large facial expression databases. Further, we show that CVML can be applied to individual human judges to infer which facial actions they use to generate perceptual emotion ratings from facial expressions.https://doi.org/10.1371/journal.pone.0211735 |
| spellingShingle | Nathaniel Haines Matthew W Southward Jennifer S Cheavens Theodore Beauchaine Woo-Young Ahn Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity. PLoS ONE |
| title | Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity. |
| title_full | Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity. |
| title_fullStr | Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity. |
| title_full_unstemmed | Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity. |
| title_short | Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity. |
| title_sort | using computer vision and machine learning to automate facial coding of positive and negative affect intensity |
| url | https://doi.org/10.1371/journal.pone.0211735 |
| work_keys_str_mv | AT nathanielhaines usingcomputervisionandmachinelearningtoautomatefacialcodingofpositiveandnegativeaffectintensity AT matthewwsouthward usingcomputervisionandmachinelearningtoautomatefacialcodingofpositiveandnegativeaffectintensity AT jenniferscheavens usingcomputervisionandmachinelearningtoautomatefacialcodingofpositiveandnegativeaffectintensity AT theodorebeauchaine usingcomputervisionandmachinelearningtoautomatefacialcodingofpositiveandnegativeaffectintensity AT wooyoungahn usingcomputervisionandmachinelearningtoautomatefacialcodingofpositiveandnegativeaffectintensity |