Decoupling facial motion features and identity features for micro-expression recognition
Background Micro-expression is a kind of expression produced by people spontaneously and unconsciously when receiving stimulus. It has the characteristics of low intensity and short duration. Moreover, it cannot be controlled and disguised. Thus, micro-expression can objectively reflect people’s rea...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2022-11-01
|
| Series: | PeerJ Computer Science |
| Subjects: | |
| Online Access: | https://peerj.com/articles/cs-1140.pdf |
| _version_ | 1798017244447375360 |
|---|---|
| author | Tingxuan Xie Guoquan Sun Hao Sun Qiang Lin Xianye Ben |
| author_facet | Tingxuan Xie Guoquan Sun Hao Sun Qiang Lin Xianye Ben |
| author_sort | Tingxuan Xie |
| collection | DOAJ |
| description | Background Micro-expression is a kind of expression produced by people spontaneously and unconsciously when receiving stimulus. It has the characteristics of low intensity and short duration. Moreover, it cannot be controlled and disguised. Thus, micro-expression can objectively reflect people’s real emotional states. Therefore, automatic recognition of micro-expressions can help machines better understand the users’ emotion, which can promote human-computer interaction. What’s more, micro-expression recognition has a wide range of applications in fields like security systems and psychological treatment. Nowadays, thanks to the development of artificial intelligence, most micro-expression recognition algorithms are based on deep learning. The features extracted by deep learning model from the micro-expression video sequences mainly contain facial motion feature information and identity feature information. However, in micro-expression recognition tasks, the motions of facial muscles are subtle. As a result, the recognition can be easily interfered by identity feature information. Methods To solve the above problem, a micro-expression recognition algorithm which decouples facial motion features and identity features is proposed in this paper. A Micro-Expression Motion Information Features Extraction Network (MENet) and an Identity Information Features Extraction Network (IDNet) are designed. By adding a Diverse Attention Operation (DAO) module and constructing divergence loss function in MENet, facial motion features can be effectively extracted. Global attention operations are used in IDNet to extract identity features. A Mutual Information Neural Estimator (MINE) is utilized to decouple facial motion features and identity features, which can help the model obtain more discriminative micro-expression features. Results Experiments on the SDU, MMEW, SAMM and CASME II datasets were conducted, which achieved competitive results and proved the superiority of the proposed algorithm. |
| first_indexed | 2024-04-11T16:03:56Z |
| format | Article |
| id | doaj.art-a35cb8462aa34db38bd54087587a59a7 |
| institution | Directory Open Access Journal |
| issn | 2376-5992 |
| language | English |
| last_indexed | 2024-04-11T16:03:56Z |
| publishDate | 2022-11-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ Computer Science |
| spelling | doaj.art-a35cb8462aa34db38bd54087587a59a72022-12-22T04:14:53ZengPeerJ Inc.PeerJ Computer Science2376-59922022-11-018e114010.7717/peerj-cs.1140Decoupling facial motion features and identity features for micro-expression recognitionTingxuan XieGuoquan SunHao SunQiang LinXianye BenBackground Micro-expression is a kind of expression produced by people spontaneously and unconsciously when receiving stimulus. It has the characteristics of low intensity and short duration. Moreover, it cannot be controlled and disguised. Thus, micro-expression can objectively reflect people’s real emotional states. Therefore, automatic recognition of micro-expressions can help machines better understand the users’ emotion, which can promote human-computer interaction. What’s more, micro-expression recognition has a wide range of applications in fields like security systems and psychological treatment. Nowadays, thanks to the development of artificial intelligence, most micro-expression recognition algorithms are based on deep learning. The features extracted by deep learning model from the micro-expression video sequences mainly contain facial motion feature information and identity feature information. However, in micro-expression recognition tasks, the motions of facial muscles are subtle. As a result, the recognition can be easily interfered by identity feature information. Methods To solve the above problem, a micro-expression recognition algorithm which decouples facial motion features and identity features is proposed in this paper. A Micro-Expression Motion Information Features Extraction Network (MENet) and an Identity Information Features Extraction Network (IDNet) are designed. By adding a Diverse Attention Operation (DAO) module and constructing divergence loss function in MENet, facial motion features can be effectively extracted. Global attention operations are used in IDNet to extract identity features. A Mutual Information Neural Estimator (MINE) is utilized to decouple facial motion features and identity features, which can help the model obtain more discriminative micro-expression features. Results Experiments on the SDU, MMEW, SAMM and CASME II datasets were conducted, which achieved competitive results and proved the superiority of the proposed algorithm.https://peerj.com/articles/cs-1140.pdfMicro-expression recognitionDeep learningFeature decouplingFacial motion featuresIdentity features |
| spellingShingle | Tingxuan Xie Guoquan Sun Hao Sun Qiang Lin Xianye Ben Decoupling facial motion features and identity features for micro-expression recognition PeerJ Computer Science Micro-expression recognition Deep learning Feature decoupling Facial motion features Identity features |
| title | Decoupling facial motion features and identity features for micro-expression recognition |
| title_full | Decoupling facial motion features and identity features for micro-expression recognition |
| title_fullStr | Decoupling facial motion features and identity features for micro-expression recognition |
| title_full_unstemmed | Decoupling facial motion features and identity features for micro-expression recognition |
| title_short | Decoupling facial motion features and identity features for micro-expression recognition |
| title_sort | decoupling facial motion features and identity features for micro expression recognition |
| topic | Micro-expression recognition Deep learning Feature decoupling Facial motion features Identity features |
| url | https://peerj.com/articles/cs-1140.pdf |
| work_keys_str_mv | AT tingxuanxie decouplingfacialmotionfeaturesandidentityfeaturesformicroexpressionrecognition AT guoquansun decouplingfacialmotionfeaturesandidentityfeaturesformicroexpressionrecognition AT haosun decouplingfacialmotionfeaturesandidentityfeaturesformicroexpressionrecognition AT qianglin decouplingfacialmotionfeaturesandidentityfeaturesformicroexpressionrecognition AT xianyeben decouplingfacialmotionfeaturesandidentityfeaturesformicroexpressionrecognition |