A lightweight method for face expression recognition based on improved MobileNetV3
Abstract Facial expression recognition plays a significant role in the application of man–machine interaction. However, existing models typically have shortcomings with numerous parameters, large model sizes, and high computational costs, which are difficult to deploy in resource‐constrained devices...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-06-01
|
| Series: | IET Image Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ipr2.12798 |
| _version_ | 1797813784327225344 |
|---|---|
| author | Xunru Liang Jianfeng Liang Tao Yin Xiaoyu Tang |
| author_facet | Xunru Liang Jianfeng Liang Tao Yin Xiaoyu Tang |
| author_sort | Xunru Liang |
| collection | DOAJ |
| description | Abstract Facial expression recognition plays a significant role in the application of man–machine interaction. However, existing models typically have shortcomings with numerous parameters, large model sizes, and high computational costs, which are difficult to deploy in resource‐constrained devices. This paper proposes a lightweight network based on improved MobileNetV3 to mitigate these disadvantages. Firstly, we adjust the channels in the high‐level network to reduce the number of parameters and model size, and then, the coordinate attention mechanism is introduced to the network, which enhances the attention of the network with few parameters and low computing cost. Furthermore, a complementary pooling structure is designed to improve the coordinate attention mechanism, which enables it to assist the network in extracting salient features sufficiently. In addition, the network with the joint loss consisting of the softmax loss and centre loss is trained, which can minimize the intra‐class gap and improve the classification performance. Finally, the network is trained and tested on public datasets FERPlus and RAF‐DB, with the best accuracy of 87.5% and 86.6%, respectively. The FLOPs, parameters, and the memory storage size are only 0.19GMac, 1.3 M, and 15.9 MB, respectively, which is lighter than most state‐of‐the‐art networks. Code is available at https://github.com/RIS‐LAB1/FER‐mobilenet. |
| first_indexed | 2024-03-13T07:57:54Z |
| format | Article |
| id | doaj.art-38df37a8176e47e294d79776f3506e19 |
| institution | Directory Open Access Journal |
| issn | 1751-9659 1751-9667 |
| language | English |
| last_indexed | 2024-03-13T07:57:54Z |
| publishDate | 2023-06-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Image Processing |
| spelling | doaj.art-38df37a8176e47e294d79776f3506e192023-06-02T03:06:38ZengWileyIET Image Processing1751-96591751-96672023-06-011782375238410.1049/ipr2.12798A lightweight method for face expression recognition based on improved MobileNetV3Xunru Liang0Jianfeng Liang1Tao Yin2Xiaoyu Tang3School of Physics and Telecommunication Engineering South China Normal University Guangzhou ChinaSchool of Physics and Telecommunication Engineering South China Normal University Guangzhou ChinaSchool of Physics and Telecommunication Engineering South China Normal University Guangzhou ChinaSchool of Physics and Telecommunication Engineering South China Normal University Guangzhou ChinaAbstract Facial expression recognition plays a significant role in the application of man–machine interaction. However, existing models typically have shortcomings with numerous parameters, large model sizes, and high computational costs, which are difficult to deploy in resource‐constrained devices. This paper proposes a lightweight network based on improved MobileNetV3 to mitigate these disadvantages. Firstly, we adjust the channels in the high‐level network to reduce the number of parameters and model size, and then, the coordinate attention mechanism is introduced to the network, which enhances the attention of the network with few parameters and low computing cost. Furthermore, a complementary pooling structure is designed to improve the coordinate attention mechanism, which enables it to assist the network in extracting salient features sufficiently. In addition, the network with the joint loss consisting of the softmax loss and centre loss is trained, which can minimize the intra‐class gap and improve the classification performance. Finally, the network is trained and tested on public datasets FERPlus and RAF‐DB, with the best accuracy of 87.5% and 86.6%, respectively. The FLOPs, parameters, and the memory storage size are only 0.19GMac, 1.3 M, and 15.9 MB, respectively, which is lighter than most state‐of‐the‐art networks. Code is available at https://github.com/RIS‐LAB1/FER‐mobilenet.https://doi.org/10.1049/ipr2.12798computer visionemotion recognitionimage classificationimage recognition |
| spellingShingle | Xunru Liang Jianfeng Liang Tao Yin Xiaoyu Tang A lightweight method for face expression recognition based on improved MobileNetV3 IET Image Processing computer vision emotion recognition image classification image recognition |
| title | A lightweight method for face expression recognition based on improved MobileNetV3 |
| title_full | A lightweight method for face expression recognition based on improved MobileNetV3 |
| title_fullStr | A lightweight method for face expression recognition based on improved MobileNetV3 |
| title_full_unstemmed | A lightweight method for face expression recognition based on improved MobileNetV3 |
| title_short | A lightweight method for face expression recognition based on improved MobileNetV3 |
| title_sort | lightweight method for face expression recognition based on improved mobilenetv3 |
| topic | computer vision emotion recognition image classification image recognition |
| url | https://doi.org/10.1049/ipr2.12798 |
| work_keys_str_mv | AT xunruliang alightweightmethodforfaceexpressionrecognitionbasedonimprovedmobilenetv3 AT jianfengliang alightweightmethodforfaceexpressionrecognitionbasedonimprovedmobilenetv3 AT taoyin alightweightmethodforfaceexpressionrecognitionbasedonimprovedmobilenetv3 AT xiaoyutang alightweightmethodforfaceexpressionrecognitionbasedonimprovedmobilenetv3 AT xunruliang lightweightmethodforfaceexpressionrecognitionbasedonimprovedmobilenetv3 AT jianfengliang lightweightmethodforfaceexpressionrecognitionbasedonimprovedmobilenetv3 AT taoyin lightweightmethodforfaceexpressionrecognitionbasedonimprovedmobilenetv3 AT xiaoyutang lightweightmethodforfaceexpressionrecognitionbasedonimprovedmobilenetv3 |