Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions
As people communicate with each other, they use gestures and facial expressions as a means to convey and understand emotional state. Non-verbal means of communication are essential to understanding, based on external clues to a person’s emotional state. Recently, active studies have been conducted o...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-04-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/10/8/2956 |
| _version_ | 1797569790358847488 |
|---|---|
| author | Chang-Min Kim Ellen J. Hong Kyungyong Chung Roy C. Park |
| author_facet | Chang-Min Kim Ellen J. Hong Kyungyong Chung Roy C. Park |
| author_sort | Chang-Min Kim |
| collection | DOAJ |
| description | As people communicate with each other, they use gestures and facial expressions as a means to convey and understand emotional state. Non-verbal means of communication are essential to understanding, based on external clues to a person’s emotional state. Recently, active studies have been conducted on the lifecare service of analyzing users’ facial expressions. Yet, rather than a service necessary for everyday life, the service is currently provided only for health care centers or certain medical institutions. It is necessary to conduct studies to prevent accidents that suddenly occur in everyday life and to cope with emergencies. Thus, we propose facial expression analysis using line-segment feature analysis-convolutional recurrent neural network (LFA-CRNN) feature extraction for health-risk assessments of drivers. The purpose of such an analysis is to manage and monitor patients with chronic diseases who are rapidly increasing in number. To prevent automobile accidents and to respond to emergency situations due to acute diseases, we propose a service that monitors a driver’s facial expressions to assess health risks and alert the driver to risk-related matters while driving. To identify health risks, deep learning technology is used to recognize expressions of pain and to determine if a person is in pain while driving. Since the amount of input-image data is large, analyzing facial expressions accurately is difficult for a process with limited resources while providing the service on a real-time basis. Accordingly, a line-segment feature analysis algorithm is proposed to reduce the amount of data, and the LFA-CRNN model was designed for this purpose. Through this model, the severity of a driver’s pain is classified into one of nine types. The LFA-CRNN model consists of one convolution layer that is reshaped and delivered into two bidirectional gated recurrent unit layers. Finally, biometric data are classified through softmax. In addition, to evaluate the performance of LFA-CRNN, the performance was compared through the CRNN and AlexNet Models based on the University of Northern British Columbia and McMaster University (UNBC-McMaster) database. |
| first_indexed | 2024-03-10T20:15:39Z |
| format | Article |
| id | doaj.art-97cd62aaf51d49b2a27a1a3ba72e1292 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T20:15:39Z |
| publishDate | 2020-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-97cd62aaf51d49b2a27a1a3ba72e12922023-11-19T22:34:29ZengMDPI AGApplied Sciences2076-34172020-04-01108295610.3390/app10082956Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk DecisionsChang-Min Kim0Ellen J. Hong1Kyungyong Chung2Roy C. Park3Division of Computer Information and Engineering, Sangji University, Wonju 26339, KoreaDepartment of Computer and Telecommunications Engineering, Yonsei University, Wonju 26493, KoreaDivision of Computer Science and Engineering, Kyonggi University, Suwon 16227, KoreaDepartment of Information Communication Software Engineering, Sangji University, Wonju 26339, KoreaAs people communicate with each other, they use gestures and facial expressions as a means to convey and understand emotional state. Non-verbal means of communication are essential to understanding, based on external clues to a person’s emotional state. Recently, active studies have been conducted on the lifecare service of analyzing users’ facial expressions. Yet, rather than a service necessary for everyday life, the service is currently provided only for health care centers or certain medical institutions. It is necessary to conduct studies to prevent accidents that suddenly occur in everyday life and to cope with emergencies. Thus, we propose facial expression analysis using line-segment feature analysis-convolutional recurrent neural network (LFA-CRNN) feature extraction for health-risk assessments of drivers. The purpose of such an analysis is to manage and monitor patients with chronic diseases who are rapidly increasing in number. To prevent automobile accidents and to respond to emergency situations due to acute diseases, we propose a service that monitors a driver’s facial expressions to assess health risks and alert the driver to risk-related matters while driving. To identify health risks, deep learning technology is used to recognize expressions of pain and to determine if a person is in pain while driving. Since the amount of input-image data is large, analyzing facial expressions accurately is difficult for a process with limited resources while providing the service on a real-time basis. Accordingly, a line-segment feature analysis algorithm is proposed to reduce the amount of data, and the LFA-CRNN model was designed for this purpose. Through this model, the severity of a driver’s pain is classified into one of nine types. The LFA-CRNN model consists of one convolution layer that is reshaped and delivered into two bidirectional gated recurrent unit layers. Finally, biometric data are classified through softmax. In addition, to evaluate the performance of LFA-CRNN, the performance was compared through the CRNN and AlexNet Models based on the University of Northern British Columbia and McMaster University (UNBC-McMaster) database.https://www.mdpi.com/2076-3417/10/8/2956facial expression analysisline segment feature analysisdimensionality reductionconvolutional recurrent neural networkdriver health risk |
| spellingShingle | Chang-Min Kim Ellen J. Hong Kyungyong Chung Roy C. Park Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions Applied Sciences facial expression analysis line segment feature analysis dimensionality reduction convolutional recurrent neural network driver health risk |
| title | Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions |
| title_full | Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions |
| title_fullStr | Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions |
| title_full_unstemmed | Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions |
| title_short | Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions |
| title_sort | driver facial expression analysis using lfa crnn based feature extraction for health risk decisions |
| topic | facial expression analysis line segment feature analysis dimensionality reduction convolutional recurrent neural network driver health risk |
| url | https://www.mdpi.com/2076-3417/10/8/2956 |
| work_keys_str_mv | AT changminkim driverfacialexpressionanalysisusinglfacrnnbasedfeatureextractionforhealthriskdecisions AT ellenjhong driverfacialexpressionanalysisusinglfacrnnbasedfeatureextractionforhealthriskdecisions AT kyungyongchung driverfacialexpressionanalysisusinglfacrnnbasedfeatureextractionforhealthriskdecisions AT roycpark driverfacialexpressionanalysisusinglfacrnnbasedfeatureextractionforhealthriskdecisions |