A novel driver emotion recognition system based on deep ensemble classification
Abstract Driver emotion classification is an important topic that can raise awareness of driving habits because many drivers are overconfident and unaware of their bad driving habits. Drivers will acquire insight into their poor driving behaviors and be better able to avoid future accidents if their...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2023-06-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-023-01100-9 |
| _version_ | 1797349566766383104 |
|---|---|
| author | Khalid Zaman Sun Zhaoyun Babar Shah Tariq Hussain Sayyed Mudassar Shah Farman Ali Umer Sadiq Khan |
| author_facet | Khalid Zaman Sun Zhaoyun Babar Shah Tariq Hussain Sayyed Mudassar Shah Farman Ali Umer Sadiq Khan |
| author_sort | Khalid Zaman |
| collection | DOAJ |
| description | Abstract Driver emotion classification is an important topic that can raise awareness of driving habits because many drivers are overconfident and unaware of their bad driving habits. Drivers will acquire insight into their poor driving behaviors and be better able to avoid future accidents if their behavior is automatically identified. In this paper, we use different models such as convolutional neural networks, recurrent neural networks, and multi-layer perceptron classification models to construct an ensemble convolutional neural network-based enhanced driver facial expression recognition model. First, the faces of the drivers are discovered using the faster region-based convolutional neural network (R-CNN) model, which can recognize faces in real-time and offline video reliably and effectively. The feature-fusing technique is utilized to integrate the features extracted from three CNN models, and the fused features are then used to train the suggested ensemble classification model. To increase the accuracy and efficiency of face detection, a new convolutional neural network block (InceptionV3) replaces the improved Faster R-CNN feature-learning block. To evaluate the proposed face detection and driver facial expression recognition (DFER) datasets, we achieved an accuracy of 98.01%, 99.53%, 99.27%, 96.81%, and 99.90% on the JAFFE, CK+, FER-2013, AffectNet, and custom-developed datasets, respectively. The custom-developed dataset has been recorded as the best among all under the simulation environment. |
| first_indexed | 2024-03-08T12:32:09Z |
| format | Article |
| id | doaj.art-413469993d824cb1b7ffb793bf932e49 |
| institution | Directory Open Access Journal |
| issn | 2199-4536 2198-6053 |
| language | English |
| last_indexed | 2024-03-08T12:32:09Z |
| publishDate | 2023-06-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj.art-413469993d824cb1b7ffb793bf932e492024-01-21T12:40:31ZengSpringerComplex & Intelligent Systems2199-45362198-60532023-06-01966927695210.1007/s40747-023-01100-9A novel driver emotion recognition system based on deep ensemble classificationKhalid Zaman0Sun Zhaoyun1Babar Shah2Tariq Hussain3Sayyed Mudassar Shah4Farman Ali5Umer Sadiq Khan6Information Engineering School, Chang’an UniversityInformation Engineering School, Chang’an UniversityCollege of Technological Innovation, Zayed UniversitySchool of Computer Science and Technology, Zhejiang Gongshang UniversityInformation Engineering School, Chang’an UniversityDepartment of Computer Science and Engineering, School of Convergence, College of Computing and Informatics, Sungkyunkwan UniversitySchool of Computer and Information Science, Hubei Engineering UniversityAbstract Driver emotion classification is an important topic that can raise awareness of driving habits because many drivers are overconfident and unaware of their bad driving habits. Drivers will acquire insight into their poor driving behaviors and be better able to avoid future accidents if their behavior is automatically identified. In this paper, we use different models such as convolutional neural networks, recurrent neural networks, and multi-layer perceptron classification models to construct an ensemble convolutional neural network-based enhanced driver facial expression recognition model. First, the faces of the drivers are discovered using the faster region-based convolutional neural network (R-CNN) model, which can recognize faces in real-time and offline video reliably and effectively. The feature-fusing technique is utilized to integrate the features extracted from three CNN models, and the fused features are then used to train the suggested ensemble classification model. To increase the accuracy and efficiency of face detection, a new convolutional neural network block (InceptionV3) replaces the improved Faster R-CNN feature-learning block. To evaluate the proposed face detection and driver facial expression recognition (DFER) datasets, we achieved an accuracy of 98.01%, 99.53%, 99.27%, 96.81%, and 99.90% on the JAFFE, CK+, FER-2013, AffectNet, and custom-developed datasets, respectively. The custom-developed dataset has been recorded as the best among all under the simulation environment.https://doi.org/10.1007/s40747-023-01100-9Driver facial expression recognition (DFER)Custom developed datasets (CDD)Computer visionAttention mechanism and DenseNetFE |
| spellingShingle | Khalid Zaman Sun Zhaoyun Babar Shah Tariq Hussain Sayyed Mudassar Shah Farman Ali Umer Sadiq Khan A novel driver emotion recognition system based on deep ensemble classification Complex & Intelligent Systems Driver facial expression recognition (DFER) Custom developed datasets (CDD) Computer vision Attention mechanism and DenseNet FE |
| title | A novel driver emotion recognition system based on deep ensemble classification |
| title_full | A novel driver emotion recognition system based on deep ensemble classification |
| title_fullStr | A novel driver emotion recognition system based on deep ensemble classification |
| title_full_unstemmed | A novel driver emotion recognition system based on deep ensemble classification |
| title_short | A novel driver emotion recognition system based on deep ensemble classification |
| title_sort | novel driver emotion recognition system based on deep ensemble classification |
| topic | Driver facial expression recognition (DFER) Custom developed datasets (CDD) Computer vision Attention mechanism and DenseNet FE |
| url | https://doi.org/10.1007/s40747-023-01100-9 |
| work_keys_str_mv | AT khalidzaman anoveldriveremotionrecognitionsystembasedondeepensembleclassification AT sunzhaoyun anoveldriveremotionrecognitionsystembasedondeepensembleclassification AT babarshah anoveldriveremotionrecognitionsystembasedondeepensembleclassification AT tariqhussain anoveldriveremotionrecognitionsystembasedondeepensembleclassification AT sayyedmudassarshah anoveldriveremotionrecognitionsystembasedondeepensembleclassification AT farmanali anoveldriveremotionrecognitionsystembasedondeepensembleclassification AT umersadiqkhan anoveldriveremotionrecognitionsystembasedondeepensembleclassification AT khalidzaman noveldriveremotionrecognitionsystembasedondeepensembleclassification AT sunzhaoyun noveldriveremotionrecognitionsystembasedondeepensembleclassification AT babarshah noveldriveremotionrecognitionsystembasedondeepensembleclassification AT tariqhussain noveldriveremotionrecognitionsystembasedondeepensembleclassification AT sayyedmudassarshah noveldriveremotionrecognitionsystembasedondeepensembleclassification AT farmanali noveldriveremotionrecognitionsystembasedondeepensembleclassification AT umersadiqkhan noveldriveremotionrecognitionsystembasedondeepensembleclassification |