Facial Expression Recognition in the Wild Using Face Graph and Attention
Facial expression recognition (FER) in the wild from various viewpoints, lighting conditions, face poses, scales, and occlusions is an extremely challenging field of research. In this study, we construct a face graph by selecting action units that play an important role in changing facial expression...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10153584/ |
| _version_ | 1797798171087208448 |
|---|---|
| author | Hyeongjin Kim Jong-Ha Lee Byoung Chul Ko |
| author_facet | Hyeongjin Kim Jong-Ha Lee Byoung Chul Ko |
| author_sort | Hyeongjin Kim |
| collection | DOAJ |
| description | Facial expression recognition (FER) in the wild from various viewpoints, lighting conditions, face poses, scales, and occlusions is an extremely challenging field of research. In this study, we construct a face graph by selecting action units that play an important role in changing facial expressions, and we propose an algorithm for recognizing facial expressions using a graph convolutional network (GCN). We first generated an attention map that can highlight action units to extract important facial expression features from faces in the wild. After feature extraction, a face graph is constructed by combining the attention map with face patches, and changes in expression in the wild are recognized using a GCN. Through comparative experiments conducted using both lab-controlled and wild datasets, we prove that the proposed method is the most suitable FER approach for use with image datasets captured in the wild and those under well-controlled indoor conditions. |
| first_indexed | 2024-03-13T03:59:30Z |
| format | Article |
| id | doaj.art-e33a22acde77489ca34237c9727af87e |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-13T03:59:30Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e33a22acde77489ca34237c9727af87e2023-06-21T23:00:22ZengIEEEIEEE Access2169-35362023-01-0111597745978710.1109/ACCESS.2023.328654710153584Facial Expression Recognition in the Wild Using Face Graph and AttentionHyeongjin Kim0Jong-Ha Lee1https://orcid.org/0000-0001-6819-8816Byoung Chul Ko2https://orcid.org/0000-0002-7284-0768Department of Computer Engineering, Keimyung University, Daegu, South KoreaDepartment of Biomedical Engineering, Keimyung University, Daegu, South KoreaDepartment of Computer Engineering, Keimyung University, Daegu, South KoreaFacial expression recognition (FER) in the wild from various viewpoints, lighting conditions, face poses, scales, and occlusions is an extremely challenging field of research. In this study, we construct a face graph by selecting action units that play an important role in changing facial expressions, and we propose an algorithm for recognizing facial expressions using a graph convolutional network (GCN). We first generated an attention map that can highlight action units to extract important facial expression features from faces in the wild. After feature extraction, a face graph is constructed by combining the attention map with face patches, and changes in expression in the wild are recognized using a GCN. Through comparative experiments conducted using both lab-controlled and wild datasets, we prove that the proposed method is the most suitable FER approach for use with image datasets captured in the wild and those under well-controlled indoor conditions.https://ieeexplore.ieee.org/document/10153584/Facial expression recognitionaction unitattention mapface graphgraph convolutional network |
| spellingShingle | Hyeongjin Kim Jong-Ha Lee Byoung Chul Ko Facial Expression Recognition in the Wild Using Face Graph and Attention IEEE Access Facial expression recognition action unit attention map face graph graph convolutional network |
| title | Facial Expression Recognition in the Wild Using Face Graph and Attention |
| title_full | Facial Expression Recognition in the Wild Using Face Graph and Attention |
| title_fullStr | Facial Expression Recognition in the Wild Using Face Graph and Attention |
| title_full_unstemmed | Facial Expression Recognition in the Wild Using Face Graph and Attention |
| title_short | Facial Expression Recognition in the Wild Using Face Graph and Attention |
| title_sort | facial expression recognition in the wild using face graph and attention |
| topic | Facial expression recognition action unit attention map face graph graph convolutional network |
| url | https://ieeexplore.ieee.org/document/10153584/ |
| work_keys_str_mv | AT hyeongjinkim facialexpressionrecognitioninthewildusingfacegraphandattention AT jonghalee facialexpressionrecognitioninthewildusingfacegraphandattention AT byoungchulko facialexpressionrecognitioninthewildusingfacegraphandattention |