Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information
In recent times, as interest in stress control has increased, many studies on stress recognition have been conducted. Several studies have been based on physiological signals, but the disadvantage of this strategy is that it requires physiological-signal-acquisition devices. Another strategy employs...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/22/7498 |
| _version_ | 1797508557055197184 |
|---|---|
| author | Taejae Jeon Han Byeol Bae Yongju Lee Sungjun Jang Sangyoun Lee |
| author_facet | Taejae Jeon Han Byeol Bae Yongju Lee Sungjun Jang Sangyoun Lee |
| author_sort | Taejae Jeon |
| collection | DOAJ |
| description | In recent times, as interest in stress control has increased, many studies on stress recognition have been conducted. Several studies have been based on physiological signals, but the disadvantage of this strategy is that it requires physiological-signal-acquisition devices. Another strategy employs facial-image-based stress-recognition methods, which do not require devices, but predominantly use handcrafted features. However, such features have low discriminating power. We propose a deep-learning-based stress-recognition method using facial images to address these challenges. Given that deep-learning methods require extensive data, we constructed a large-capacity image database for stress recognition. Furthermore, we used temporal attention, which assigns a high weight to frames that are highly related to stress, as well as spatial attention, which assigns a high weight to regions that are highly related to stress. By adding a network that inputs the facial landmark information closely related to stress, we supplemented the network that receives only facial images as the input. Experimental results on our newly constructed database indicated that the proposed method outperforms contemporary deep-learning-based recognition methods. |
| first_indexed | 2024-03-10T05:05:37Z |
| format | Article |
| id | doaj.art-fd4838c6636545f797504a666d47f8e5 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T05:05:37Z |
| publishDate | 2021-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-fd4838c6636545f797504a666d47f8e52023-11-23T01:24:18ZengMDPI AGSensors1424-82202021-11-012122749810.3390/s21227498Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial InformationTaejae Jeon0Han Byeol Bae1Yongju Lee2Sungjun Jang3Sangyoun Lee4Department of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, KoreaDepartment of Artificial Intelligence Convergence, Kwangju Women’s University, 45 Yeodae-gil, Gwangsan-gu, Gwangju 62396, KoreaDepartment of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, KoreaDepartment of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, KoreaDepartment of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, KoreaIn recent times, as interest in stress control has increased, many studies on stress recognition have been conducted. Several studies have been based on physiological signals, but the disadvantage of this strategy is that it requires physiological-signal-acquisition devices. Another strategy employs facial-image-based stress-recognition methods, which do not require devices, but predominantly use handcrafted features. However, such features have low discriminating power. We propose a deep-learning-based stress-recognition method using facial images to address these challenges. Given that deep-learning methods require extensive data, we constructed a large-capacity image database for stress recognition. Furthermore, we used temporal attention, which assigns a high weight to frames that are highly related to stress, as well as spatial attention, which assigns a high weight to regions that are highly related to stress. By adding a network that inputs the facial landmark information closely related to stress, we supplemented the network that receives only facial images as the input. Experimental results on our newly constructed database indicated that the proposed method outperforms contemporary deep-learning-based recognition methods.https://www.mdpi.com/1424-8220/21/22/7498deep learningstress recognitionstress databasespatial attentiontemporal attentionfacial landmark |
| spellingShingle | Taejae Jeon Han Byeol Bae Yongju Lee Sungjun Jang Sangyoun Lee Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information Sensors deep learning stress recognition stress database spatial attention temporal attention facial landmark |
| title | Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information |
| title_full | Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information |
| title_fullStr | Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information |
| title_full_unstemmed | Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information |
| title_short | Deep-Learning-Based Stress Recognition with Spatial-Temporal Facial Information |
| title_sort | deep learning based stress recognition with spatial temporal facial information |
| topic | deep learning stress recognition stress database spatial attention temporal attention facial landmark |
| url | https://www.mdpi.com/1424-8220/21/22/7498 |
| work_keys_str_mv | AT taejaejeon deeplearningbasedstressrecognitionwithspatialtemporalfacialinformation AT hanbyeolbae deeplearningbasedstressrecognitionwithspatialtemporalfacialinformation AT yongjulee deeplearningbasedstressrecognitionwithspatialtemporalfacialinformation AT sungjunjang deeplearningbasedstressrecognitionwithspatialtemporalfacialinformation AT sangyounlee deeplearningbasedstressrecognitionwithspatialtemporalfacialinformation |