A study on computer vision for facial emotion recognition
Abstract Artificial intelligence has been successfully applied in various fields, one of which is computer vision. In this study, a deep neural network (DNN) was adopted for Facial emotion recognition (FER). One of the objectives in this study is to identify the critical facial features on which the...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-05-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-023-35446-4 |
| _version_ | 1797818048811368448 |
|---|---|
| author | Zi-Yu Huang Chia-Chin Chiang Jian-Hao Chen Yi-Chian Chen Hsin-Lung Chung Yu-Ping Cai Hsiu-Chuan Hsu |
| author_facet | Zi-Yu Huang Chia-Chin Chiang Jian-Hao Chen Yi-Chian Chen Hsin-Lung Chung Yu-Ping Cai Hsiu-Chuan Hsu |
| author_sort | Zi-Yu Huang |
| collection | DOAJ |
| description | Abstract Artificial intelligence has been successfully applied in various fields, one of which is computer vision. In this study, a deep neural network (DNN) was adopted for Facial emotion recognition (FER). One of the objectives in this study is to identify the critical facial features on which the DNN model focuses for FER. In particular, we utilized a convolutional neural network (CNN), the combination of squeeze-and-excitation network and the residual neural network, for the task of FER. We utilized AffectNet and the Real-World Affective Faces Database (RAF-DB) as the facial expression databases that provide learning samples for the CNN. The feature maps were extracted from the residual blocks for further analysis. Our analysis shows that the features around the nose and mouth are critical facial landmarks for the neural networks. Cross-database validations were conducted between the databases. The network model trained on AffectNet achieved 77.37% accuracy when validated on the RAF-DB, while the network model pretrained on AffectNet and then transfer learned on the RAF-DB results in validation accuracy of 83.37%. The outcomes of this study would improve the understanding of neural networks and assist with improving computer vision accuracy. |
| first_indexed | 2024-03-13T09:02:25Z |
| format | Article |
| id | doaj.art-2cd60fc5f8c74457b9cec4986324ba1a |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-03-13T09:02:25Z |
| publishDate | 2023-05-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-2cd60fc5f8c74457b9cec4986324ba1a2023-05-28T11:13:28ZengNature PortfolioScientific Reports2045-23222023-05-0113111310.1038/s41598-023-35446-4A study on computer vision for facial emotion recognitionZi-Yu Huang0Chia-Chin Chiang1Jian-Hao Chen2Yi-Chian Chen3Hsin-Lung Chung4Yu-Ping Cai5Hsiu-Chuan Hsu6Department of Mechanical Engineering, National Kaohsiung University of Science and TechnologyDepartment of Mechanical Engineering, National Kaohsiung University of Science and TechnologyGraduate Institute of Applied Physics, National Chengchi UniversityDepartment of Occupational Safety and Hygiene, Fooyin UniversityDepartment of Mechanical Engineering, National Kaohsiung University of Science and TechnologyDepartment of Nursing, Hsin Sheng Junior College of Medical Care and ManagementGraduate Institute of Applied Physics, National Chengchi UniversityAbstract Artificial intelligence has been successfully applied in various fields, one of which is computer vision. In this study, a deep neural network (DNN) was adopted for Facial emotion recognition (FER). One of the objectives in this study is to identify the critical facial features on which the DNN model focuses for FER. In particular, we utilized a convolutional neural network (CNN), the combination of squeeze-and-excitation network and the residual neural network, for the task of FER. We utilized AffectNet and the Real-World Affective Faces Database (RAF-DB) as the facial expression databases that provide learning samples for the CNN. The feature maps were extracted from the residual blocks for further analysis. Our analysis shows that the features around the nose and mouth are critical facial landmarks for the neural networks. Cross-database validations were conducted between the databases. The network model trained on AffectNet achieved 77.37% accuracy when validated on the RAF-DB, while the network model pretrained on AffectNet and then transfer learned on the RAF-DB results in validation accuracy of 83.37%. The outcomes of this study would improve the understanding of neural networks and assist with improving computer vision accuracy.https://doi.org/10.1038/s41598-023-35446-4 |
| spellingShingle | Zi-Yu Huang Chia-Chin Chiang Jian-Hao Chen Yi-Chian Chen Hsin-Lung Chung Yu-Ping Cai Hsiu-Chuan Hsu A study on computer vision for facial emotion recognition Scientific Reports |
| title | A study on computer vision for facial emotion recognition |
| title_full | A study on computer vision for facial emotion recognition |
| title_fullStr | A study on computer vision for facial emotion recognition |
| title_full_unstemmed | A study on computer vision for facial emotion recognition |
| title_short | A study on computer vision for facial emotion recognition |
| title_sort | study on computer vision for facial emotion recognition |
| url | https://doi.org/10.1038/s41598-023-35446-4 |
| work_keys_str_mv | AT ziyuhuang astudyoncomputervisionforfacialemotionrecognition AT chiachinchiang astudyoncomputervisionforfacialemotionrecognition AT jianhaochen astudyoncomputervisionforfacialemotionrecognition AT yichianchen astudyoncomputervisionforfacialemotionrecognition AT hsinlungchung astudyoncomputervisionforfacialemotionrecognition AT yupingcai astudyoncomputervisionforfacialemotionrecognition AT hsiuchuanhsu astudyoncomputervisionforfacialemotionrecognition AT ziyuhuang studyoncomputervisionforfacialemotionrecognition AT chiachinchiang studyoncomputervisionforfacialemotionrecognition AT jianhaochen studyoncomputervisionforfacialemotionrecognition AT yichianchen studyoncomputervisionforfacialemotionrecognition AT hsinlungchung studyoncomputervisionforfacialemotionrecognition AT yupingcai studyoncomputervisionforfacialemotionrecognition AT hsiuchuanhsu studyoncomputervisionforfacialemotionrecognition |