Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition
A micro-expression is defined as an uncontrollable muscular movement shown on the face of humans when one is trying to conceal or repress his true emotions. Many researchers have applied the deep learning framework to micro-expression recognition in recent years. However, few have introduced the hum...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-07-01
|
| Series: | Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2078-2489/11/8/380 |
| _version_ | 1797561027686039552 |
|---|---|
| author | Boyu Chen Zhihao Zhang Nian Liu Yang Tan Xinyu Liu Tong Chen |
| author_facet | Boyu Chen Zhihao Zhang Nian Liu Yang Tan Xinyu Liu Tong Chen |
| author_sort | Boyu Chen |
| collection | DOAJ |
| description | A micro-expression is defined as an uncontrollable muscular movement shown on the face of humans when one is trying to conceal or repress his true emotions. Many researchers have applied the deep learning framework to micro-expression recognition in recent years. However, few have introduced the human visual attention mechanism to micro-expression recognition. In this study, we propose a three-dimensional (3D) spatiotemporal convolutional neural network with the convolutional block attention module (CBAM) for micro-expression recognition. First image sequences were input to a medium-sized convolutional neural network (CNN) to extract visual features. Afterwards, it learned to allocate the feature weights in an adaptive manner with the help of a convolutional block attention module. The method was testified in spontaneous micro-expression databases (Chinese Academy of Sciences Micro-expression II (CASME II), Spontaneous Micro-expression Database (SMIC)). The experimental results show that the 3D CNN with convolutional block attention module outperformed other algorithms in micro-expression recognition. |
| first_indexed | 2024-03-10T18:09:31Z |
| format | Article |
| id | doaj.art-ce9cc5675cc04e5fa53187b50febc9c4 |
| institution | Directory Open Access Journal |
| issn | 2078-2489 |
| language | English |
| last_indexed | 2024-03-10T18:09:31Z |
| publishDate | 2020-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Information |
| spelling | doaj.art-ce9cc5675cc04e5fa53187b50febc9c42023-11-20T08:17:40ZengMDPI AGInformation2078-24892020-07-0111838010.3390/info11080380Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression RecognitionBoyu Chen0Zhihao Zhang1Nian Liu2Yang Tan3Xinyu Liu4Tong Chen5School of Electronic and Information Engineering, Southwest University, Chongqing 400715, ChinaSchool of Electronic and Information Engineering, Southwest University, Chongqing 400715, ChinaSchool of Electronic and Information Engineering, Southwest University, Chongqing 400715, ChinaSchool of Electronic and Information Engineering, Southwest University, Chongqing 400715, ChinaSchool of Electronic and Information Engineering, Southwest University, Chongqing 400715, ChinaSchool of Electronic and Information Engineering, Southwest University, Chongqing 400715, ChinaA micro-expression is defined as an uncontrollable muscular movement shown on the face of humans when one is trying to conceal or repress his true emotions. Many researchers have applied the deep learning framework to micro-expression recognition in recent years. However, few have introduced the human visual attention mechanism to micro-expression recognition. In this study, we propose a three-dimensional (3D) spatiotemporal convolutional neural network with the convolutional block attention module (CBAM) for micro-expression recognition. First image sequences were input to a medium-sized convolutional neural network (CNN) to extract visual features. Afterwards, it learned to allocate the feature weights in an adaptive manner with the help of a convolutional block attention module. The method was testified in spontaneous micro-expression databases (Chinese Academy of Sciences Micro-expression II (CASME II), Spontaneous Micro-expression Database (SMIC)). The experimental results show that the 3D CNN with convolutional block attention module outperformed other algorithms in micro-expression recognition.https://www.mdpi.com/2078-2489/11/8/380micro-expression recognition3D convolutional neural network (3D CNN)convolutional block attention module (CBAM)adaptive feature weightsspatiotemporal features |
| spellingShingle | Boyu Chen Zhihao Zhang Nian Liu Yang Tan Xinyu Liu Tong Chen Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition Information micro-expression recognition 3D convolutional neural network (3D CNN) convolutional block attention module (CBAM) adaptive feature weights spatiotemporal features |
| title | Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition |
| title_full | Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition |
| title_fullStr | Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition |
| title_full_unstemmed | Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition |
| title_short | Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition |
| title_sort | spatiotemporal convolutional neural network with convolutional block attention module for micro expression recognition |
| topic | micro-expression recognition 3D convolutional neural network (3D CNN) convolutional block attention module (CBAM) adaptive feature weights spatiotemporal features |
| url | https://www.mdpi.com/2078-2489/11/8/380 |
| work_keys_str_mv | AT boyuchen spatiotemporalconvolutionalneuralnetworkwithconvolutionalblockattentionmoduleformicroexpressionrecognition AT zhihaozhang spatiotemporalconvolutionalneuralnetworkwithconvolutionalblockattentionmoduleformicroexpressionrecognition AT nianliu spatiotemporalconvolutionalneuralnetworkwithconvolutionalblockattentionmoduleformicroexpressionrecognition AT yangtan spatiotemporalconvolutionalneuralnetworkwithconvolutionalblockattentionmoduleformicroexpressionrecognition AT xinyuliu spatiotemporalconvolutionalneuralnetworkwithconvolutionalblockattentionmoduleformicroexpressionrecognition AT tongchen spatiotemporalconvolutionalneuralnetworkwithconvolutionalblockattentionmoduleformicroexpressionrecognition |