Dual-ATME: Dual-Branch Attention Network for Micro-Expression Recognition
Micro-expression recognition (MER) is challenging due to the difficulty of capturing the instantaneous and subtle motion changes of micro-expressions (MEs). Early works based on hand-crafted features extracted from prior knowledge showed some promising results, but have recently been replaced by dee...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-03-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/25/3/460 |
| _version_ | 1797611985202839552 |
|---|---|
| author | Haoliang Zhou Shucheng Huang Jingting Li Su-Jing Wang |
| author_facet | Haoliang Zhou Shucheng Huang Jingting Li Su-Jing Wang |
| author_sort | Haoliang Zhou |
| collection | DOAJ |
| description | Micro-expression recognition (MER) is challenging due to the difficulty of capturing the instantaneous and subtle motion changes of micro-expressions (MEs). Early works based on hand-crafted features extracted from prior knowledge showed some promising results, but have recently been replaced by deep learning methods based on the attention mechanism. However, with limited ME sample sizes, features extracted by these methods lack discriminative ME representations, in yet-to-be improved MER performance. This paper proposes the Dual-branch Attention Network (Dual-ATME) for MER to address the problem of ineffective single-scale features representing MEs. Specifically, Dual-ATME consists of two components: Hand-crafted Attention Region Selection (HARS) and Automated Attention Region Selection (AARS). HARS uses prior knowledge to manually extract features from regions of interest (ROIs). Meanwhile, AARS is based on attention mechanisms and extracts hidden information from data automatically. Finally, through similarity comparison and feature fusion, the dual-scale features could be used to learn ME representations effectively. Experiments on spontaneous ME datasets (including CASME II, SAMM, SMIC) and their composite dataset, MEGC2019-CD, showed that Dual-ATME achieves better, or more competitive, performance than the state-of-the-art MER methods. |
| first_indexed | 2024-03-11T06:35:04Z |
| format | Article |
| id | doaj.art-884bb89e21a448ec887ac35bb7939df1 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-11T06:35:04Z |
| publishDate | 2023-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-884bb89e21a448ec887ac35bb7939df12023-11-17T10:56:31ZengMDPI AGEntropy1099-43002023-03-0125346010.3390/e25030460Dual-ATME: Dual-Branch Attention Network for Micro-Expression RecognitionHaoliang Zhou0Shucheng Huang1Jingting Li2Su-Jing Wang3School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, ChinaSchool of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, ChinaKey Laboratory of Behavior Sciences, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, ChinaKey Laboratory of Behavior Sciences, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101, ChinaMicro-expression recognition (MER) is challenging due to the difficulty of capturing the instantaneous and subtle motion changes of micro-expressions (MEs). Early works based on hand-crafted features extracted from prior knowledge showed some promising results, but have recently been replaced by deep learning methods based on the attention mechanism. However, with limited ME sample sizes, features extracted by these methods lack discriminative ME representations, in yet-to-be improved MER performance. This paper proposes the Dual-branch Attention Network (Dual-ATME) for MER to address the problem of ineffective single-scale features representing MEs. Specifically, Dual-ATME consists of two components: Hand-crafted Attention Region Selection (HARS) and Automated Attention Region Selection (AARS). HARS uses prior knowledge to manually extract features from regions of interest (ROIs). Meanwhile, AARS is based on attention mechanisms and extracts hidden information from data automatically. Finally, through similarity comparison and feature fusion, the dual-scale features could be used to learn ME representations effectively. Experiments on spontaneous ME datasets (including CASME II, SAMM, SMIC) and their composite dataset, MEGC2019-CD, showed that Dual-ATME achieves better, or more competitive, performance than the state-of-the-art MER methods.https://www.mdpi.com/1099-4300/25/3/460micro-expression recognitionattention mechanismregions of interest |
| spellingShingle | Haoliang Zhou Shucheng Huang Jingting Li Su-Jing Wang Dual-ATME: Dual-Branch Attention Network for Micro-Expression Recognition Entropy micro-expression recognition attention mechanism regions of interest |
| title | Dual-ATME: Dual-Branch Attention Network for Micro-Expression Recognition |
| title_full | Dual-ATME: Dual-Branch Attention Network for Micro-Expression Recognition |
| title_fullStr | Dual-ATME: Dual-Branch Attention Network for Micro-Expression Recognition |
| title_full_unstemmed | Dual-ATME: Dual-Branch Attention Network for Micro-Expression Recognition |
| title_short | Dual-ATME: Dual-Branch Attention Network for Micro-Expression Recognition |
| title_sort | dual atme dual branch attention network for micro expression recognition |
| topic | micro-expression recognition attention mechanism regions of interest |
| url | https://www.mdpi.com/1099-4300/25/3/460 |
| work_keys_str_mv | AT haoliangzhou dualatmedualbranchattentionnetworkformicroexpressionrecognition AT shuchenghuang dualatmedualbranchattentionnetworkformicroexpressionrecognition AT jingtingli dualatmedualbranchattentionnetworkformicroexpressionrecognition AT sujingwang dualatmedualbranchattentionnetworkformicroexpressionrecognition |