mFERMeta++ : Robust Multiview Facial Expression Recognition Based on Metahuman and Metalearning
Facial pose variation presents a significant challenge to facial expression recognition (FER) in real‐world applications. Significant bottlenecks exist in the field of multiview facial expression recognition (MFER) including a lack of high‐quality MFER datasets, and the limited model robustness in r...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-10-01
|
| Series: | Advanced Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/aisy.202300210 |
| _version_ | 1827788047344730112 |
|---|---|
| author | Shuo Jiang Jiahang Liu Yanmin Zhou Zhipeng Wang Bin He |
| author_facet | Shuo Jiang Jiahang Liu Yanmin Zhou Zhipeng Wang Bin He |
| author_sort | Shuo Jiang |
| collection | DOAJ |
| description | Facial pose variation presents a significant challenge to facial expression recognition (FER) in real‐world applications. Significant bottlenecks exist in the field of multiview facial expression recognition (MFER) including a lack of high‐quality MFER datasets, and the limited model robustness in real‐world MFER scenarios. Therefore, this article first introduces a metahuman‐based MFER dataset (MMED), which effectively addresses the issues of insufficient quantity and quality in existing datasets. Second, a conditional cascade VGG (ccVGG) model is proposed, which can adaptively adjust expression feature extraction based on the input image's pose information. Finally, a hybrid training and few‐shot learning strategy are proposed that integrates our MMED dataset with a real‐world dataset and quickly deploys it in real‐world application scenarios using the proposed Meta‐Dist few‐shot learning method. Experiments on the Karolinska Directed Emotional Face (KDEF) dataset demonstrate that the proposed model exhibits improved robustness in multiview application scenarios and achieves a recognition accuracy improvement of 28.68% relative to the baseline. It demonstrates that the proposed MMED dataset can effectively improve the training efficiency of MFER models and facilitate easy deployment in real‐world applications. This work provides a reliable dataset for the MFER studies and paves the way for robust FER in any view for real‐world deployment. |
| first_indexed | 2024-03-11T17:01:53Z |
| format | Article |
| id | doaj.art-ffefc707e68c4cb187142f8d704e34b4 |
| institution | Directory Open Access Journal |
| issn | 2640-4567 |
| language | English |
| last_indexed | 2024-03-11T17:01:53Z |
| publishDate | 2023-10-01 |
| publisher | Wiley |
| record_format | Article |
| series | Advanced Intelligent Systems |
| spelling | doaj.art-ffefc707e68c4cb187142f8d704e34b42023-10-20T07:43:39ZengWileyAdvanced Intelligent Systems2640-45672023-10-01510n/an/a10.1002/aisy.202300210mFERMeta++ : Robust Multiview Facial Expression Recognition Based on Metahuman and MetalearningShuo Jiang0Jiahang Liu1Yanmin Zhou2Zhipeng Wang3Bin He4College of Electronics and Information Engineering Tongji University Shanghai 201804 ChinaCollege of Electronics and Information Engineering Tongji University Shanghai 201804 ChinaCollege of Electronics and Information Engineering Tongji University Shanghai 201804 ChinaCollege of Electronics and Information Engineering Tongji University Shanghai 201804 ChinaCollege of Electronics and Information Engineering Tongji University Shanghai 201804 ChinaFacial pose variation presents a significant challenge to facial expression recognition (FER) in real‐world applications. Significant bottlenecks exist in the field of multiview facial expression recognition (MFER) including a lack of high‐quality MFER datasets, and the limited model robustness in real‐world MFER scenarios. Therefore, this article first introduces a metahuman‐based MFER dataset (MMED), which effectively addresses the issues of insufficient quantity and quality in existing datasets. Second, a conditional cascade VGG (ccVGG) model is proposed, which can adaptively adjust expression feature extraction based on the input image's pose information. Finally, a hybrid training and few‐shot learning strategy are proposed that integrates our MMED dataset with a real‐world dataset and quickly deploys it in real‐world application scenarios using the proposed Meta‐Dist few‐shot learning method. Experiments on the Karolinska Directed Emotional Face (KDEF) dataset demonstrate that the proposed model exhibits improved robustness in multiview application scenarios and achieves a recognition accuracy improvement of 28.68% relative to the baseline. It demonstrates that the proposed MMED dataset can effectively improve the training efficiency of MFER models and facilitate easy deployment in real‐world applications. This work provides a reliable dataset for the MFER studies and paves the way for robust FER in any view for real‐world deployment.https://doi.org/10.1002/aisy.202300210few-shot learningmetahumanmultiview facial expression recognitionpose-robust |
| spellingShingle | Shuo Jiang Jiahang Liu Yanmin Zhou Zhipeng Wang Bin He mFERMeta++ : Robust Multiview Facial Expression Recognition Based on Metahuman and Metalearning Advanced Intelligent Systems few-shot learning metahuman multiview facial expression recognition pose-robust |
| title | mFERMeta++ : Robust Multiview Facial Expression Recognition Based on Metahuman and Metalearning |
| title_full | mFERMeta++ : Robust Multiview Facial Expression Recognition Based on Metahuman and Metalearning |
| title_fullStr | mFERMeta++ : Robust Multiview Facial Expression Recognition Based on Metahuman and Metalearning |
| title_full_unstemmed | mFERMeta++ : Robust Multiview Facial Expression Recognition Based on Metahuman and Metalearning |
| title_short | mFERMeta++ : Robust Multiview Facial Expression Recognition Based on Metahuman and Metalearning |
| title_sort | mfermeta robust multiview facial expression recognition based on metahuman and metalearning |
| topic | few-shot learning metahuman multiview facial expression recognition pose-robust |
| url | https://doi.org/10.1002/aisy.202300210 |
| work_keys_str_mv | AT shuojiang mfermetarobustmultiviewfacialexpressionrecognitionbasedonmetahumanandmetalearning AT jiahangliu mfermetarobustmultiviewfacialexpressionrecognitionbasedonmetahumanandmetalearning AT yanminzhou mfermetarobustmultiviewfacialexpressionrecognitionbasedonmetahumanandmetalearning AT zhipengwang mfermetarobustmultiviewfacialexpressionrecognitionbasedonmetahumanandmetalearning AT binhe mfermetarobustmultiviewfacialexpressionrecognitionbasedonmetahumanandmetalearning |