Driver’s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion
Head pose and eye gaze are vital clues for analysing a driver’s visual attention. Previous approaches achieve promising results from point clouds in constrained conditions. However, these approaches face challenges in the complex naturalistic driving scene. One of the challenges is that the collecte...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/9/3154 |
| _version_ | 1797502897030692864 |
|---|---|
| author | Yafei Wang Guoliang Yuan Xianping Fu |
| author_facet | Yafei Wang Guoliang Yuan Xianping Fu |
| author_sort | Yafei Wang |
| collection | DOAJ |
| description | Head pose and eye gaze are vital clues for analysing a driver’s visual attention. Previous approaches achieve promising results from point clouds in constrained conditions. However, these approaches face challenges in the complex naturalistic driving scene. One of the challenges is that the collected point cloud data under non-uniform illumination and large head rotation is prone to partial facial occlusion. It causes bad transformation during failed template matching or incorrect feature extraction. In this paper, a novel estimation method is proposed for predicting accurate driver head pose and gaze zone using an RGB-D camera, with an effective point cloud fusion and registration strategy. In the fusion step, to reduce bad transformation, continuous multi-frame point clouds are registered and fused to generate a stable point cloud. In the registration step, to reduce reliance on template registration, multiple point clouds in the nearest neighbor gaze zone are utilized as a template point cloud. A coarse transformation computed by the normal distributions transform is used as the initial transformation, and updated with particle filter. A gaze zone estimator is trained by combining the head pose and eye image features, in which the head pose is predicted by point cloud registration, and the eye image features are extracted via multi-scale spare coding. Extensive experiments demonstrate that the proposed strategy achieves better results on head pose tracking, and also has a low error on gaze zone classification. |
| first_indexed | 2024-03-10T03:42:42Z |
| format | Article |
| id | doaj.art-201cfae820ed4042877f19be949f32c3 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T03:42:42Z |
| publishDate | 2022-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-201cfae820ed4042877f19be949f32c32023-11-23T09:14:13ZengMDPI AGSensors1424-82202022-04-01229315410.3390/s22093154Driver’s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud FusionYafei Wang0Guoliang Yuan1Xianping Fu2School of Information Science and Technology, Dalian Maritime University, Dalian 116026, ChinaSchool of Information Science and Technology, Dalian Maritime University, Dalian 116026, ChinaSchool of Information Science and Technology, Dalian Maritime University, Dalian 116026, ChinaHead pose and eye gaze are vital clues for analysing a driver’s visual attention. Previous approaches achieve promising results from point clouds in constrained conditions. However, these approaches face challenges in the complex naturalistic driving scene. One of the challenges is that the collected point cloud data under non-uniform illumination and large head rotation is prone to partial facial occlusion. It causes bad transformation during failed template matching or incorrect feature extraction. In this paper, a novel estimation method is proposed for predicting accurate driver head pose and gaze zone using an RGB-D camera, with an effective point cloud fusion and registration strategy. In the fusion step, to reduce bad transformation, continuous multi-frame point clouds are registered and fused to generate a stable point cloud. In the registration step, to reduce reliance on template registration, multiple point clouds in the nearest neighbor gaze zone are utilized as a template point cloud. A coarse transformation computed by the normal distributions transform is used as the initial transformation, and updated with particle filter. A gaze zone estimator is trained by combining the head pose and eye image features, in which the head pose is predicted by point cloud registration, and the eye image features are extracted via multi-scale spare coding. Extensive experiments demonstrate that the proposed strategy achieves better results on head pose tracking, and also has a low error on gaze zone classification.https://www.mdpi.com/1424-8220/22/9/3154driving environmenthead poseICPpoint cloudgaze zone |
| spellingShingle | Yafei Wang Guoliang Yuan Xianping Fu Driver’s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion Sensors driving environment head pose ICP point cloud gaze zone |
| title | Driver’s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion |
| title_full | Driver’s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion |
| title_fullStr | Driver’s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion |
| title_full_unstemmed | Driver’s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion |
| title_short | Driver’s Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion |
| title_sort | driver s head pose and gaze zone estimation based on multi zone templates registration and multi frame point cloud fusion |
| topic | driving environment head pose ICP point cloud gaze zone |
| url | https://www.mdpi.com/1424-8220/22/9/3154 |
| work_keys_str_mv | AT yafeiwang driversheadposeandgazezoneestimationbasedonmultizonetemplatesregistrationandmultiframepointcloudfusion AT guoliangyuan driversheadposeandgazezoneestimationbasedonmultizonetemplatesregistrationandmultiframepointcloudfusion AT xianpingfu driversheadposeandgazezoneestimationbasedonmultizonetemplatesregistrationandmultiframepointcloudfusion |