A Multi-Semantic Driver Behavior Recognition Model of Autonomous Vehicles Using Confidence Fusion Mechanism
With the rise of autonomous vehicles, drivers are gradually being liberated from the traditional roles behind steering wheels. Driver behavior cognition is significant for improving safety, comfort, and human–vehicle interaction. Existing research mostly analyzes driver behaviors relying on the move...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-08-01
|
| Series: | Actuators |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-0825/10/9/218 |
| _version_ | 1829453716280836096 |
|---|---|
| author | Hongze Ren Yage Guo Zhonghao Bai Xiangyu Cheng |
| author_facet | Hongze Ren Yage Guo Zhonghao Bai Xiangyu Cheng |
| author_sort | Hongze Ren |
| collection | DOAJ |
| description | With the rise of autonomous vehicles, drivers are gradually being liberated from the traditional roles behind steering wheels. Driver behavior cognition is significant for improving safety, comfort, and human–vehicle interaction. Existing research mostly analyzes driver behaviors relying on the movements of upper-body parts, which may lead to false positives and missed detections due to the subtle changes among similar behaviors. In this paper, an end-to-end model is proposed to tackle the problem of the accurate classification of similar driver actions in real-time, known as MSRNet. The proposed architecture is made up of two major branches: the action detection network and the object detection network, which can extract spatiotemporal and key-object features, respectively. Then, the confidence fusion mechanism is introduced to aggregate the predictions from both branches based on the semantic relationships between actions and key objects. Experiments implemented on the modified version of the public dataset Drive&Act demonstrate that the MSRNet can recognize 11 different behaviors with 64.18% accuracy and a 20 fps inference time on an 8-frame input clip. Compared to the state-of-the-art action recognition model, our approach obtains higher accuracy, especially for behaviors with similar movements. |
| first_indexed | 2024-03-10T08:00:02Z |
| format | Article |
| id | doaj.art-5f4342d97cde4e128b3862a3ef21abc9 |
| institution | Directory Open Access Journal |
| issn | 2076-0825 |
| language | English |
| last_indexed | 2024-03-10T08:00:02Z |
| publishDate | 2021-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Actuators |
| spelling | doaj.art-5f4342d97cde4e128b3862a3ef21abc92023-11-22T11:33:00ZengMDPI AGActuators2076-08252021-08-0110921810.3390/act10090218A Multi-Semantic Driver Behavior Recognition Model of Autonomous Vehicles Using Confidence Fusion MechanismHongze Ren0Yage Guo1Zhonghao Bai2Xiangyu Cheng3The State Key Laboratory of Advanced Design and Manufacturing for Vehicle Body, Hunan University, Changsha 410082, ChinaThe State Key Laboratory of Advanced Design and Manufacturing for Vehicle Body, Hunan University, Changsha 410082, ChinaThe State Key Laboratory of Advanced Design and Manufacturing for Vehicle Body, Hunan University, Changsha 410082, ChinaThe State Key Laboratory of Advanced Design and Manufacturing for Vehicle Body, Hunan University, Changsha 410082, ChinaWith the rise of autonomous vehicles, drivers are gradually being liberated from the traditional roles behind steering wheels. Driver behavior cognition is significant for improving safety, comfort, and human–vehicle interaction. Existing research mostly analyzes driver behaviors relying on the movements of upper-body parts, which may lead to false positives and missed detections due to the subtle changes among similar behaviors. In this paper, an end-to-end model is proposed to tackle the problem of the accurate classification of similar driver actions in real-time, known as MSRNet. The proposed architecture is made up of two major branches: the action detection network and the object detection network, which can extract spatiotemporal and key-object features, respectively. Then, the confidence fusion mechanism is introduced to aggregate the predictions from both branches based on the semantic relationships between actions and key objects. Experiments implemented on the modified version of the public dataset Drive&Act demonstrate that the MSRNet can recognize 11 different behaviors with 64.18% accuracy and a 20 fps inference time on an 8-frame input clip. Compared to the state-of-the-art action recognition model, our approach obtains higher accuracy, especially for behaviors with similar movements.https://www.mdpi.com/2076-0825/10/9/218intelligent electric vehiclesdriver behavior recognitionmulti-semantic descriptionconfidence fusion |
| spellingShingle | Hongze Ren Yage Guo Zhonghao Bai Xiangyu Cheng A Multi-Semantic Driver Behavior Recognition Model of Autonomous Vehicles Using Confidence Fusion Mechanism Actuators intelligent electric vehicles driver behavior recognition multi-semantic description confidence fusion |
| title | A Multi-Semantic Driver Behavior Recognition Model of Autonomous Vehicles Using Confidence Fusion Mechanism |
| title_full | A Multi-Semantic Driver Behavior Recognition Model of Autonomous Vehicles Using Confidence Fusion Mechanism |
| title_fullStr | A Multi-Semantic Driver Behavior Recognition Model of Autonomous Vehicles Using Confidence Fusion Mechanism |
| title_full_unstemmed | A Multi-Semantic Driver Behavior Recognition Model of Autonomous Vehicles Using Confidence Fusion Mechanism |
| title_short | A Multi-Semantic Driver Behavior Recognition Model of Autonomous Vehicles Using Confidence Fusion Mechanism |
| title_sort | multi semantic driver behavior recognition model of autonomous vehicles using confidence fusion mechanism |
| topic | intelligent electric vehicles driver behavior recognition multi-semantic description confidence fusion |
| url | https://www.mdpi.com/2076-0825/10/9/218 |
| work_keys_str_mv | AT hongzeren amultisemanticdriverbehaviorrecognitionmodelofautonomousvehiclesusingconfidencefusionmechanism AT yageguo amultisemanticdriverbehaviorrecognitionmodelofautonomousvehiclesusingconfidencefusionmechanism AT zhonghaobai amultisemanticdriverbehaviorrecognitionmodelofautonomousvehiclesusingconfidencefusionmechanism AT xiangyucheng amultisemanticdriverbehaviorrecognitionmodelofautonomousvehiclesusingconfidencefusionmechanism AT hongzeren multisemanticdriverbehaviorrecognitionmodelofautonomousvehiclesusingconfidencefusionmechanism AT yageguo multisemanticdriverbehaviorrecognitionmodelofautonomousvehiclesusingconfidencefusionmechanism AT zhonghaobai multisemanticdriverbehaviorrecognitionmodelofautonomousvehiclesusingconfidencefusionmechanism AT xiangyucheng multisemanticdriverbehaviorrecognitionmodelofautonomousvehiclesusingconfidencefusionmechanism |