Research on real-time teachers’ facial expression recognition based on YOLOv5 and attention mechanisms
Abstract Studying the real-time face expression state of teachers in class was important to build an objective classroom teaching evaluation system based on AI. However, the face-to-face communication in classroom conditions was a real-time process that operated on a millisecond time scale. Therefor...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2023-05-01
|
| Series: | EURASIP Journal on Advances in Signal Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13634-023-01019-w |
| _version_ | 1797827292500590592 |
|---|---|
| author | Hongmei Zhong Tingting Han Wei Xia Yan Tian Libao Wu |
| author_facet | Hongmei Zhong Tingting Han Wei Xia Yan Tian Libao Wu |
| author_sort | Hongmei Zhong |
| collection | DOAJ |
| description | Abstract Studying the real-time face expression state of teachers in class was important to build an objective classroom teaching evaluation system based on AI. However, the face-to-face communication in classroom conditions was a real-time process that operated on a millisecond time scale. Therefore, in order to quickly and accurately predict teachers’ facial expressions in real time, this paper proposed an improved YOLOv5 network, which introduced the attention mechanisms into the Backbone model of YOLOv5. In experiments, we investigated the effects of different attention mechanisms on YOLOv5 by adding different attention mechanisms after each CBS module in the CSP1_X structure of the Backbone part, respectively. At the same time, the attention mechanisms were incorporated at different locations of the Focus, CBS, and SPP modules of YOLOv5, respectively, to study the effects of the attention mechanism on different modules. The results showed that the network in which the coordinate attentions were incorporated after each CBS module in the CSP1_X structure obtained the detection time of 25 ms and the accuracy of 77.1% which increased by 3.5% compared with YOLOv5. It outperformed other networks, including Faster-RCNN, R-FCN, ResNext-101, DETR, Swin-Transformer, YOLOv3, and YOLOX. Finally, the real-time teachers’ facial expression recognition system was designed to detect and analyze the teachers’ facial expression distribution with time through camera and the teaching video. |
| first_indexed | 2024-04-09T12:46:08Z |
| format | Article |
| id | doaj.art-59dbdbee253a4d368190f199c2d4bacf |
| institution | Directory Open Access Journal |
| issn | 1687-6180 |
| language | English |
| last_indexed | 2024-04-09T12:46:08Z |
| publishDate | 2023-05-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | EURASIP Journal on Advances in Signal Processing |
| spelling | doaj.art-59dbdbee253a4d368190f199c2d4bacf2023-05-14T11:31:21ZengSpringerOpenEURASIP Journal on Advances in Signal Processing1687-61802023-05-012023111510.1186/s13634-023-01019-wResearch on real-time teachers’ facial expression recognition based on YOLOv5 and attention mechanismsHongmei Zhong0Tingting Han1Wei Xia2Yan Tian3Libao Wu4Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal UniversityTianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal UniversityTianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal UniversityTianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal UniversityFaculty of Education, Tianjin Normal UniversityAbstract Studying the real-time face expression state of teachers in class was important to build an objective classroom teaching evaluation system based on AI. However, the face-to-face communication in classroom conditions was a real-time process that operated on a millisecond time scale. Therefore, in order to quickly and accurately predict teachers’ facial expressions in real time, this paper proposed an improved YOLOv5 network, which introduced the attention mechanisms into the Backbone model of YOLOv5. In experiments, we investigated the effects of different attention mechanisms on YOLOv5 by adding different attention mechanisms after each CBS module in the CSP1_X structure of the Backbone part, respectively. At the same time, the attention mechanisms were incorporated at different locations of the Focus, CBS, and SPP modules of YOLOv5, respectively, to study the effects of the attention mechanism on different modules. The results showed that the network in which the coordinate attentions were incorporated after each CBS module in the CSP1_X structure obtained the detection time of 25 ms and the accuracy of 77.1% which increased by 3.5% compared with YOLOv5. It outperformed other networks, including Faster-RCNN, R-FCN, ResNext-101, DETR, Swin-Transformer, YOLOv3, and YOLOX. Finally, the real-time teachers’ facial expression recognition system was designed to detect and analyze the teachers’ facial expression distribution with time through camera and the teaching video.https://doi.org/10.1186/s13634-023-01019-wTeachers’ facial expression recognitionYOLOv5Attention mechanismEducation |
| spellingShingle | Hongmei Zhong Tingting Han Wei Xia Yan Tian Libao Wu Research on real-time teachers’ facial expression recognition based on YOLOv5 and attention mechanisms EURASIP Journal on Advances in Signal Processing Teachers’ facial expression recognition YOLOv5 Attention mechanism Education |
| title | Research on real-time teachers’ facial expression recognition based on YOLOv5 and attention mechanisms |
| title_full | Research on real-time teachers’ facial expression recognition based on YOLOv5 and attention mechanisms |
| title_fullStr | Research on real-time teachers’ facial expression recognition based on YOLOv5 and attention mechanisms |
| title_full_unstemmed | Research on real-time teachers’ facial expression recognition based on YOLOv5 and attention mechanisms |
| title_short | Research on real-time teachers’ facial expression recognition based on YOLOv5 and attention mechanisms |
| title_sort | research on real time teachers facial expression recognition based on yolov5 and attention mechanisms |
| topic | Teachers’ facial expression recognition YOLOv5 Attention mechanism Education |
| url | https://doi.org/10.1186/s13634-023-01019-w |
| work_keys_str_mv | AT hongmeizhong researchonrealtimeteachersfacialexpressionrecognitionbasedonyolov5andattentionmechanisms AT tingtinghan researchonrealtimeteachersfacialexpressionrecognitionbasedonyolov5andattentionmechanisms AT weixia researchonrealtimeteachersfacialexpressionrecognitionbasedonyolov5andattentionmechanisms AT yantian researchonrealtimeteachersfacialexpressionrecognitionbasedonyolov5andattentionmechanisms AT libaowu researchonrealtimeteachersfacialexpressionrecognitionbasedonyolov5andattentionmechanisms |