Improvement of continuous emotion recognition of temporal convolutional networks with incomplete labels
Abstract Video‐based emotion recognition has been a long‐standing research topic for computer scientists and psychiatrists. In contrast to traditional discrete emotional models, emotion recognition based on continuous emotional models can better describe the progression of emotions. Quantitative ana...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-03-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.12994 |
| _version_ | 1827325186604531712 |
|---|---|
| author | Zheyu Wang Jieying Zheng Feng Liu |
| author_facet | Zheyu Wang Jieying Zheng Feng Liu |
| author_sort | Zheyu Wang |
| collection | DOAJ |
| description | Abstract Video‐based emotion recognition has been a long‐standing research topic for computer scientists and psychiatrists. In contrast to traditional discrete emotional models, emotion recognition based on continuous emotional models can better describe the progression of emotions. Quantitative analysis of emotions will have crucial impacts on promoting the development of intelligent products. The current solutions to continuous emotion recognition still have many issues. The original continuous emotion dataset contains incomplete data annotations, and the existing methods often ignore temporal information between frames. The following measures are taken in response to the above problems. Initially, aiming at the problem of incomplete video labels, the correlation between discrete and continuous video emotion labels is used to complete the dataset labels. This correlation is used to propose a mathematical model to fill the missing labels of the original dataset without adding data. Moreover, this paper proposes a continuous emotion recognition network based on an optimized temporal convolutional network, which adds a feature extraction submodule and a residual module to retain shallow features while improving the feature extraction ability. Finally, validation experiments on the Aff‐wild2 dataset achieved accuracies of 0.5159 and 0.65611 on the valence and arousal dimensions, respectively, by adopting the above measures. |
| first_indexed | 2024-03-07T14:17:40Z |
| format | Article |
| id | doaj.art-9b5fb798f95d45e08bb23be88d852450 |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-03-07T14:17:40Z |
| publishDate | 2024-03-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-9b5fb798f95d45e08bb23be88d8524502024-03-06T11:42:57ZengWileyIET Image Processing1751-96591751-96672024-03-0118491492510.1049/ipr2.12994Improvement of continuous emotion recognition of temporal convolutional networks with incomplete labelsZheyu Wang0Jieying Zheng1Feng Liu2School of Communication and Information Engineering Nanjing University of Posts and Telecommunications Nanjing Jiangsu ChinaSchool of Geographic and Biologic information Nanjing University of Posts and Telecommunications Nanjing Jiangsu ChinaSchool of Communication and Information Engineering Nanjing University of Posts and Telecommunications Nanjing Jiangsu ChinaAbstract Video‐based emotion recognition has been a long‐standing research topic for computer scientists and psychiatrists. In contrast to traditional discrete emotional models, emotion recognition based on continuous emotional models can better describe the progression of emotions. Quantitative analysis of emotions will have crucial impacts on promoting the development of intelligent products. The current solutions to continuous emotion recognition still have many issues. The original continuous emotion dataset contains incomplete data annotations, and the existing methods often ignore temporal information between frames. The following measures are taken in response to the above problems. Initially, aiming at the problem of incomplete video labels, the correlation between discrete and continuous video emotion labels is used to complete the dataset labels. This correlation is used to propose a mathematical model to fill the missing labels of the original dataset without adding data. Moreover, this paper proposes a continuous emotion recognition network based on an optimized temporal convolutional network, which adds a feature extraction submodule and a residual module to retain shallow features while improving the feature extraction ability. Finally, validation experiments on the Aff‐wild2 dataset achieved accuracies of 0.5159 and 0.65611 on the valence and arousal dimensions, respectively, by adopting the above measures.https://doi.org/10.1049/ipr2.12994convolutional neural netsemotion recognition |
| spellingShingle | Zheyu Wang Jieying Zheng Feng Liu Improvement of continuous emotion recognition of temporal convolutional networks with incomplete labels IET Image Processing convolutional neural nets emotion recognition |
| title | Improvement of continuous emotion recognition of temporal convolutional networks with incomplete labels |
| title_full | Improvement of continuous emotion recognition of temporal convolutional networks with incomplete labels |
| title_fullStr | Improvement of continuous emotion recognition of temporal convolutional networks with incomplete labels |
| title_full_unstemmed | Improvement of continuous emotion recognition of temporal convolutional networks with incomplete labels |
| title_short | Improvement of continuous emotion recognition of temporal convolutional networks with incomplete labels |
| title_sort | improvement of continuous emotion recognition of temporal convolutional networks with incomplete labels |
| topic | convolutional neural nets emotion recognition |
| url | https://doi.org/10.1049/ipr2.12994 |
| work_keys_str_mv | AT zheyuwang improvementofcontinuousemotionrecognitionoftemporalconvolutionalnetworkswithincompletelabels AT jieyingzheng improvementofcontinuousemotionrecognitionoftemporalconvolutionalnetworkswithincompletelabels AT fengliu improvementofcontinuousemotionrecognitionoftemporalconvolutionalnetworkswithincompletelabels |