Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithm
Cognitive science plays a pivotal role in deciphering human behavior by understanding and interpreting emotions prevalent in everyday life. These emotions manifest through various cues, including speech patterns, body language, and notably, facial expressions. Human facial expressions serve as a fun...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-03-01
|
| Series: | Intelligent Systems with Applications |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2667305324000152 |
| _version_ | 1797287544501567488 |
|---|---|
| author | Haposan Vincentius Manalu Achmad Pratama Rifai |
| author_facet | Haposan Vincentius Manalu Achmad Pratama Rifai |
| author_sort | Haposan Vincentius Manalu |
| collection | DOAJ |
| description | Cognitive science plays a pivotal role in deciphering human behavior by understanding and interpreting emotions prevalent in everyday life. These emotions manifest through various cues, including speech patterns, body language, and notably, facial expressions. Human facial expressions serve as a fundamental mode of communication and interaction. Within the realm of computer vision, Facial Expression Recognition (FER) stands as a crucial field, offering diverse techniques to decode emotions from facial expressions. This research aims to develop a hybrid Convolutional Neural Network – Recurrent Neural Network (CNN-RNN) model adept at detecting human emotions derived from facial expressions based on video data. The models are developed based on Emotional Wearable Dataset 2020. This dataset consists of several expressions, four of them - amusement, enthusiasm, awe, and liking - has never been explored in the previous datasets. This expansion provides a more comprehensive approach to emotion detection. Three models – MobileNetV2-RNN, InceptionV3-RNN, and custom CNN-RNN – are developed for the classification. The custom CNN-RNN model achieved an accuracy rate of 63 %, while the MobileNetV2-RNN and InceptionV3-RNN transfer learning models yield 59 % and 66 %, respectively. The developed models demonstrate enhanced efficiency in distinguishing these nuanced emotions, a significant advancement in the field of facial expression recognition. This research holds substantial implications for cognitive science and real-world applications, particularly in enhancing interactive digital communication and emotional analysis. |
| first_indexed | 2024-03-07T18:34:42Z |
| format | Article |
| id | doaj.art-0ed6174ee90b46fc8753ac8689c1605e |
| institution | Directory Open Access Journal |
| issn | 2667-3053 |
| language | English |
| last_indexed | 2024-03-07T18:34:42Z |
| publishDate | 2024-03-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Intelligent Systems with Applications |
| spelling | doaj.art-0ed6174ee90b46fc8753ac8689c1605e2024-03-02T04:55:22ZengElsevierIntelligent Systems with Applications2667-30532024-03-0121200339Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithmHaposan Vincentius Manalu0Achmad Pratama Rifai1Department of Mechanical and Industrial Engineering, Universitas Gadjah Mada, Yogyakarta, IndonesiaCorresponding author.; Department of Mechanical and Industrial Engineering, Universitas Gadjah Mada, Yogyakarta, IndonesiaCognitive science plays a pivotal role in deciphering human behavior by understanding and interpreting emotions prevalent in everyday life. These emotions manifest through various cues, including speech patterns, body language, and notably, facial expressions. Human facial expressions serve as a fundamental mode of communication and interaction. Within the realm of computer vision, Facial Expression Recognition (FER) stands as a crucial field, offering diverse techniques to decode emotions from facial expressions. This research aims to develop a hybrid Convolutional Neural Network – Recurrent Neural Network (CNN-RNN) model adept at detecting human emotions derived from facial expressions based on video data. The models are developed based on Emotional Wearable Dataset 2020. This dataset consists of several expressions, four of them - amusement, enthusiasm, awe, and liking - has never been explored in the previous datasets. This expansion provides a more comprehensive approach to emotion detection. Three models – MobileNetV2-RNN, InceptionV3-RNN, and custom CNN-RNN – are developed for the classification. The custom CNN-RNN model achieved an accuracy rate of 63 %, while the MobileNetV2-RNN and InceptionV3-RNN transfer learning models yield 59 % and 66 %, respectively. The developed models demonstrate enhanced efficiency in distinguishing these nuanced emotions, a significant advancement in the field of facial expression recognition. This research holds substantial implications for cognitive science and real-world applications, particularly in enhancing interactive digital communication and emotional analysis.http://www.sciencedirect.com/science/article/pii/S2667305324000152Facial expression recognition (fer)CNN-RNNComputer VisionTransfer Learning |
| spellingShingle | Haposan Vincentius Manalu Achmad Pratama Rifai Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithm Intelligent Systems with Applications Facial expression recognition (fer) CNN-RNN Computer Vision Transfer Learning |
| title | Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithm |
| title_full | Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithm |
| title_fullStr | Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithm |
| title_full_unstemmed | Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithm |
| title_short | Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithm |
| title_sort | detection of human emotions through facial expressions using hybrid convolutional neural network recurrent neural network algorithm |
| topic | Facial expression recognition (fer) CNN-RNN Computer Vision Transfer Learning |
| url | http://www.sciencedirect.com/science/article/pii/S2667305324000152 |
| work_keys_str_mv | AT haposanvincentiusmanalu detectionofhumanemotionsthroughfacialexpressionsusinghybridconvolutionalneuralnetworkrecurrentneuralnetworkalgorithm AT achmadpratamarifai detectionofhumanemotionsthroughfacialexpressionsusinghybridconvolutionalneuralnetworkrecurrentneuralnetworkalgorithm |