The unbearable (technical) unreliability of automated facial emotion recognition
Emotion recognition, and in particular acial emotion recognition (FER), is among the most controversial applications of machine learning, not least because of its ethical implications for human subjects. In this article, we address the controversial conjecture that machines can read emotions from ou...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2022-07-01
|
| Series: | Big Data & Society |
| Online Access: | https://doi.org/10.1177/20539517221129549 |
| _version_ | 1811202465100464128 |
|---|---|
| author | Federico Cabitza Andrea Campagner Martina Mattioli |
| author_facet | Federico Cabitza Andrea Campagner Martina Mattioli |
| author_sort | Federico Cabitza |
| collection | DOAJ |
| description | Emotion recognition, and in particular acial emotion recognition (FER), is among the most controversial applications of machine learning, not least because of its ethical implications for human subjects. In this article, we address the controversial conjecture that machines can read emotions from our facial expressions by asking whether this task can be performed reliably. This means, rather than considering the potential harms or scientific soundness of facial emotion recognition systems, focusing on the reliability of the ground truths used to develop emotion recognition systems, assessing how well different human observers agree on the emotions they detect in subjects’ faces. Additionally, we discuss the extent to which sharing context can help observers agree on the emotions they perceive on subjects’ faces. Briefly, we demonstrate that when large and heterogeneous samples of observers are involved, the task of emotion detection from static images crumbles into inconsistency. We thus reveal that any endeavour to understand human behaviour from large sets of labelled patterns is over-ambitious, even if it were technically feasible. We conclude that we cannot speak of actual accuracy for facial emotion recognition systems for any practical purposes. |
| first_indexed | 2024-04-12T02:39:14Z |
| format | Article |
| id | doaj.art-ca8e646484a441b99c181427e1fcc9a2 |
| institution | Directory Open Access Journal |
| issn | 2053-9517 |
| language | English |
| last_indexed | 2024-04-12T02:39:14Z |
| publishDate | 2022-07-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Big Data & Society |
| spelling | doaj.art-ca8e646484a441b99c181427e1fcc9a22022-12-22T03:51:22ZengSAGE PublishingBig Data & Society2053-95172022-07-01910.1177/20539517221129549The unbearable (technical) unreliability of automated facial emotion recognitionFederico Cabitza0Andrea Campagner1Martina Mattioli2 , Milan, Italy University of Milan-Bicocca, Milan, Italy University of Milan-Bicocca, Milan, ItalyEmotion recognition, and in particular acial emotion recognition (FER), is among the most controversial applications of machine learning, not least because of its ethical implications for human subjects. In this article, we address the controversial conjecture that machines can read emotions from our facial expressions by asking whether this task can be performed reliably. This means, rather than considering the potential harms or scientific soundness of facial emotion recognition systems, focusing on the reliability of the ground truths used to develop emotion recognition systems, assessing how well different human observers agree on the emotions they detect in subjects’ faces. Additionally, we discuss the extent to which sharing context can help observers agree on the emotions they perceive on subjects’ faces. Briefly, we demonstrate that when large and heterogeneous samples of observers are involved, the task of emotion detection from static images crumbles into inconsistency. We thus reveal that any endeavour to understand human behaviour from large sets of labelled patterns is over-ambitious, even if it were technically feasible. We conclude that we cannot speak of actual accuracy for facial emotion recognition systems for any practical purposes.https://doi.org/10.1177/20539517221129549 |
| spellingShingle | Federico Cabitza Andrea Campagner Martina Mattioli The unbearable (technical) unreliability of automated facial emotion recognition Big Data & Society |
| title | The unbearable (technical) unreliability of automated facial emotion recognition |
| title_full | The unbearable (technical) unreliability of automated facial emotion recognition |
| title_fullStr | The unbearable (technical) unreliability of automated facial emotion recognition |
| title_full_unstemmed | The unbearable (technical) unreliability of automated facial emotion recognition |
| title_short | The unbearable (technical) unreliability of automated facial emotion recognition |
| title_sort | unbearable technical unreliability of automated facial emotion recognition |
| url | https://doi.org/10.1177/20539517221129549 |
| work_keys_str_mv | AT federicocabitza theunbearabletechnicalunreliabilityofautomatedfacialemotionrecognition AT andreacampagner theunbearabletechnicalunreliabilityofautomatedfacialemotionrecognition AT martinamattioli theunbearabletechnicalunreliabilityofautomatedfacialemotionrecognition AT federicocabitza unbearabletechnicalunreliabilityofautomatedfacialemotionrecognition AT andreacampagner unbearabletechnicalunreliabilityofautomatedfacialemotionrecognition AT martinamattioli unbearabletechnicalunreliabilityofautomatedfacialemotionrecognition |