Graph Reasoning-Based Emotion Recognition Network
Semantic information from images can be used to improve the performance of deep learning methods in recognizing human emotions. In this paper, we propose a novel framework based on the graph convolutional network for emotion recognition by utilizing the semantic relationships of different regions. F...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9312197/ |
| _version_ | 1818448937439199232 |
|---|---|
| author | Qinquan Gao Hanxin Zeng Gen Li Tong Tong |
| author_facet | Qinquan Gao Hanxin Zeng Gen Li Tong Tong |
| author_sort | Qinquan Gao |
| collection | DOAJ |
| description | Semantic information from images can be used to improve the performance of deep learning methods in recognizing human emotions. In this paper, we propose a novel framework based on the graph convolutional network for emotion recognition by utilizing the semantic relationships of different regions. First, we extract the salient image regions within video frame clips by using the bottom-up attention module to construct the node features of a graph. Then, we build the graphs containing the node features and the semantic correlations of nodes by using the graph convolutional network. For refinement, each node feature of graph vectors is enhanced via a gated recurrent unit consisting of gate and memory units to remove redundant feature information. Experimental results show that our proposed method achieves superior performance over state-of-the-art approaches for the emotion recognition on the CEAR and AFEW datasets. |
| first_indexed | 2024-12-14T20:27:27Z |
| format | Article |
| id | doaj.art-a99abfa5290a477a85d3bdf5c5dd163b |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-14T20:27:27Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a99abfa5290a477a85d3bdf5c5dd163b2022-12-21T22:48:36ZengIEEEIEEE Access2169-35362021-01-0196488649710.1109/ACCESS.2020.30486939312197Graph Reasoning-Based Emotion Recognition NetworkQinquan Gao0Hanxin Zeng1https://orcid.org/0000-0002-0153-6726Gen Li2Tong Tong3College of Physics and Information Engineering, Fuzhou University, Fuzhou, ChinaCollege of Physics and Information Engineering, Fuzhou University, Fuzhou, ChinaImperial Vision Technology, Fuzhou, ChinaCollege of Physics and Information Engineering, Fuzhou University, Fuzhou, ChinaSemantic information from images can be used to improve the performance of deep learning methods in recognizing human emotions. In this paper, we propose a novel framework based on the graph convolutional network for emotion recognition by utilizing the semantic relationships of different regions. First, we extract the salient image regions within video frame clips by using the bottom-up attention module to construct the node features of a graph. Then, we build the graphs containing the node features and the semantic correlations of nodes by using the graph convolutional network. For refinement, each node feature of graph vectors is enhanced via a gated recurrent unit consisting of gate and memory units to remove redundant feature information. Experimental results show that our proposed method achieves superior performance over state-of-the-art approaches for the emotion recognition on the CEAR and AFEW datasets.https://ieeexplore.ieee.org/document/9312197/Emotion recognitiongraph convolutional neural networkscontextual spatiotemporal features |
| spellingShingle | Qinquan Gao Hanxin Zeng Gen Li Tong Tong Graph Reasoning-Based Emotion Recognition Network IEEE Access Emotion recognition graph convolutional neural networks contextual spatiotemporal features |
| title | Graph Reasoning-Based Emotion Recognition Network |
| title_full | Graph Reasoning-Based Emotion Recognition Network |
| title_fullStr | Graph Reasoning-Based Emotion Recognition Network |
| title_full_unstemmed | Graph Reasoning-Based Emotion Recognition Network |
| title_short | Graph Reasoning-Based Emotion Recognition Network |
| title_sort | graph reasoning based emotion recognition network |
| topic | Emotion recognition graph convolutional neural networks contextual spatiotemporal features |
| url | https://ieeexplore.ieee.org/document/9312197/ |
| work_keys_str_mv | AT qinquangao graphreasoningbasedemotionrecognitionnetwork AT hanxinzeng graphreasoningbasedemotionrecognitionnetwork AT genli graphreasoningbasedemotionrecognitionnetwork AT tongtong graphreasoningbasedemotionrecognitionnetwork |