Micro-Expression Recognition Using Uncertainty-Aware Magnification-Robust Networks
A micro-expression (ME) is a kind of involuntary facial expressions, which commonly occurs with subtle intensity. The accurately recognition ME, a. k. a. micro-expression recognition (MER), has a number of potential applications, e.g., interrogation and clinical diagnosis. Therefore, the subject has...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/24/9/1271 |
| _version_ | 1797488651292114944 |
|---|---|
| author | Mengting Wei Yuan Zong Xingxun Jiang Cheng Lu Jiateng Liu |
| author_facet | Mengting Wei Yuan Zong Xingxun Jiang Cheng Lu Jiateng Liu |
| author_sort | Mengting Wei |
| collection | DOAJ |
| description | A micro-expression (ME) is a kind of involuntary facial expressions, which commonly occurs with subtle intensity. The accurately recognition ME, a. k. a. micro-expression recognition (MER), has a number of potential applications, e.g., interrogation and clinical diagnosis. Therefore, the subject has received a high level of attention among researchers in affective computing and pattern recognition communities. In this paper, we proposed a straightforward and effective deep learning method called uncertainty-aware magnification-robust networks (UAMRN) for MER, which attempts to address two key issues in MER including the low intensity of ME and imbalance of ME samples. Specifically, to better distinguish subtle ME movements, we reconstructed a new sequence by magnifying the ME intensity. Furthermore, a sparse self-attention (SSA) block was implemented which rectifies the standard self-attention with locality sensitive hashing (LSH), resulting in the suppression of artefacts generated during magnification. On the other hand, for the class imbalance problem, we guided the network optimization based on the confidence about the estimation, through which the samples from rare classes were allotted greater uncertainty and thus trained more carefully. We conducted the experiments on three public ME databases, i.e., CASME II, SAMM and SMIC-HS, the results of which demonstrate improvement compared to recent state-of-the-art MER methods. |
| first_indexed | 2024-03-10T00:05:21Z |
| format | Article |
| id | doaj.art-7a783b17772d4b65981018f2db6e9f67 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T00:05:21Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-7a783b17772d4b65981018f2db6e9f672023-11-23T16:08:50ZengMDPI AGEntropy1099-43002022-09-01249127110.3390/e24091271Micro-Expression Recognition Using Uncertainty-Aware Magnification-Robust NetworksMengting Wei0Yuan Zong1Xingxun Jiang2Cheng Lu3Jiateng Liu4Key Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University, Nanjing 210096, ChinaKey Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University, Nanjing 210096, ChinaKey Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University, Nanjing 210096, ChinaKey Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University, Nanjing 210096, ChinaKey Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University, Nanjing 210096, ChinaA micro-expression (ME) is a kind of involuntary facial expressions, which commonly occurs with subtle intensity. The accurately recognition ME, a. k. a. micro-expression recognition (MER), has a number of potential applications, e.g., interrogation and clinical diagnosis. Therefore, the subject has received a high level of attention among researchers in affective computing and pattern recognition communities. In this paper, we proposed a straightforward and effective deep learning method called uncertainty-aware magnification-robust networks (UAMRN) for MER, which attempts to address two key issues in MER including the low intensity of ME and imbalance of ME samples. Specifically, to better distinguish subtle ME movements, we reconstructed a new sequence by magnifying the ME intensity. Furthermore, a sparse self-attention (SSA) block was implemented which rectifies the standard self-attention with locality sensitive hashing (LSH), resulting in the suppression of artefacts generated during magnification. On the other hand, for the class imbalance problem, we guided the network optimization based on the confidence about the estimation, through which the samples from rare classes were allotted greater uncertainty and thus trained more carefully. We conducted the experiments on three public ME databases, i.e., CASME II, SAMM and SMIC-HS, the results of which demonstrate improvement compared to recent state-of-the-art MER methods.https://www.mdpi.com/1099-4300/24/9/1271micro-expression recognitionmicro-expression magnificationlocality sensitive hashinguncertaintyself-attention |
| spellingShingle | Mengting Wei Yuan Zong Xingxun Jiang Cheng Lu Jiateng Liu Micro-Expression Recognition Using Uncertainty-Aware Magnification-Robust Networks Entropy micro-expression recognition micro-expression magnification locality sensitive hashing uncertainty self-attention |
| title | Micro-Expression Recognition Using Uncertainty-Aware Magnification-Robust Networks |
| title_full | Micro-Expression Recognition Using Uncertainty-Aware Magnification-Robust Networks |
| title_fullStr | Micro-Expression Recognition Using Uncertainty-Aware Magnification-Robust Networks |
| title_full_unstemmed | Micro-Expression Recognition Using Uncertainty-Aware Magnification-Robust Networks |
| title_short | Micro-Expression Recognition Using Uncertainty-Aware Magnification-Robust Networks |
| title_sort | micro expression recognition using uncertainty aware magnification robust networks |
| topic | micro-expression recognition micro-expression magnification locality sensitive hashing uncertainty self-attention |
| url | https://www.mdpi.com/1099-4300/24/9/1271 |
| work_keys_str_mv | AT mengtingwei microexpressionrecognitionusinguncertaintyawaremagnificationrobustnetworks AT yuanzong microexpressionrecognitionusinguncertaintyawaremagnificationrobustnetworks AT xingxunjiang microexpressionrecognitionusinguncertaintyawaremagnificationrobustnetworks AT chenglu microexpressionrecognitionusinguncertaintyawaremagnificationrobustnetworks AT jiatengliu microexpressionrecognitionusinguncertaintyawaremagnificationrobustnetworks |