Electromyographic Validation of Spontaneous Facial Mimicry Detection Using Automated Facial Action Coding
Although electromyography (EMG) remains the standard, researchers have begun using automated facial action coding system (FACS) software to evaluate spontaneous facial mimicry despite the lack of evidence of its validity. Using the facial EMG of the zygomaticus major (ZM) as a standard, we confirmed...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/22/9076 |
| _version_ | 1797457872435544064 |
|---|---|
| author | Chun-Ting Hsu Wataru Sato |
| author_facet | Chun-Ting Hsu Wataru Sato |
| author_sort | Chun-Ting Hsu |
| collection | DOAJ |
| description | Although electromyography (EMG) remains the standard, researchers have begun using automated facial action coding system (FACS) software to evaluate spontaneous facial mimicry despite the lack of evidence of its validity. Using the facial EMG of the zygomaticus major (ZM) as a standard, we confirmed the detection of spontaneous facial mimicry in action unit 12 (AU12, lip corner puller) via an automated FACS. Participants were alternately presented with real-time model performance and prerecorded videos of dynamic facial expressions, while simultaneous ZM signal and frontal facial videos were acquired. Facial videos were estimated for AU12 using FaceReader, Py-Feat, and OpenFace. The automated FACS is less sensitive and less accurate than facial EMG, but AU12 mimicking responses were significantly correlated with ZM responses. All three software programs detected enhanced facial mimicry by live performances. The AU12 time series showed a roughly 100 to 300 ms latency relative to the ZM. Our results suggested that while the automated FACS could not replace facial EMG in mimicry detection, it could serve a purpose for large effect sizes. Researchers should be cautious with the automated FACS outputs, especially when studying clinical populations. In addition, developers should consider the EMG validation of AU estimation as a benchmark. |
| first_indexed | 2024-03-09T16:29:00Z |
| format | Article |
| id | doaj.art-fbc41d50c7ad4275b5c76603417e447e |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T16:29:00Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-fbc41d50c7ad4275b5c76603417e447e2023-11-24T15:05:15ZengMDPI AGSensors1424-82202023-11-012322907610.3390/s23229076Electromyographic Validation of Spontaneous Facial Mimicry Detection Using Automated Facial Action CodingChun-Ting Hsu0Wataru Sato1Psychological Process Research Team, Guardian Robot Project, RIKEN, Soraku-gun, Kyoto 619-0288, JapanPsychological Process Research Team, Guardian Robot Project, RIKEN, Soraku-gun, Kyoto 619-0288, JapanAlthough electromyography (EMG) remains the standard, researchers have begun using automated facial action coding system (FACS) software to evaluate spontaneous facial mimicry despite the lack of evidence of its validity. Using the facial EMG of the zygomaticus major (ZM) as a standard, we confirmed the detection of spontaneous facial mimicry in action unit 12 (AU12, lip corner puller) via an automated FACS. Participants were alternately presented with real-time model performance and prerecorded videos of dynamic facial expressions, while simultaneous ZM signal and frontal facial videos were acquired. Facial videos were estimated for AU12 using FaceReader, Py-Feat, and OpenFace. The automated FACS is less sensitive and less accurate than facial EMG, but AU12 mimicking responses were significantly correlated with ZM responses. All three software programs detected enhanced facial mimicry by live performances. The AU12 time series showed a roughly 100 to 300 ms latency relative to the ZM. Our results suggested that while the automated FACS could not replace facial EMG in mimicry detection, it could serve a purpose for large effect sizes. Researchers should be cautious with the automated FACS outputs, especially when studying clinical populations. In addition, developers should consider the EMG validation of AU estimation as a benchmark.https://www.mdpi.com/1424-8220/23/22/9076spontaneous facial mimicryelectromyographyfacial action coding systemFaceReaderPy-FeatOpenFace |
| spellingShingle | Chun-Ting Hsu Wataru Sato Electromyographic Validation of Spontaneous Facial Mimicry Detection Using Automated Facial Action Coding Sensors spontaneous facial mimicry electromyography facial action coding system FaceReader Py-Feat OpenFace |
| title | Electromyographic Validation of Spontaneous Facial Mimicry Detection Using Automated Facial Action Coding |
| title_full | Electromyographic Validation of Spontaneous Facial Mimicry Detection Using Automated Facial Action Coding |
| title_fullStr | Electromyographic Validation of Spontaneous Facial Mimicry Detection Using Automated Facial Action Coding |
| title_full_unstemmed | Electromyographic Validation of Spontaneous Facial Mimicry Detection Using Automated Facial Action Coding |
| title_short | Electromyographic Validation of Spontaneous Facial Mimicry Detection Using Automated Facial Action Coding |
| title_sort | electromyographic validation of spontaneous facial mimicry detection using automated facial action coding |
| topic | spontaneous facial mimicry electromyography facial action coding system FaceReader Py-Feat OpenFace |
| url | https://www.mdpi.com/1424-8220/23/22/9076 |
| work_keys_str_mv | AT chuntinghsu electromyographicvalidationofspontaneousfacialmimicrydetectionusingautomatedfacialactioncoding AT watarusato electromyographicvalidationofspontaneousfacialmimicrydetectionusingautomatedfacialactioncoding |