Real-Time Eye Tracking for Bare and Sunglasses-Wearing Faces for Augmented Reality 3D Head-Up Displays
Eye pupil tracking is important for augmented reality (AR) three-dimensional (3D) head-up displays (HUDs). Accurate and fast eye tracking is still challenging due to multiple driving conditions with eye occlusions, such as wearing sunglasses. In this paper, we propose a system for commercial use tha...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9530401/ |
| _version_ | 1827382778005553152 |
|---|---|
| author | Dongwoo Kang Lin Ma |
| author_facet | Dongwoo Kang Lin Ma |
| author_sort | Dongwoo Kang |
| collection | DOAJ |
| description | Eye pupil tracking is important for augmented reality (AR) three-dimensional (3D) head-up displays (HUDs). Accurate and fast eye tracking is still challenging due to multiple driving conditions with eye occlusions, such as wearing sunglasses. In this paper, we propose a system for commercial use that can handle practical driving conditions. Our system classifies human faces into bare faces and sunglasses faces, which are treated differently. For bare faces, our eye tracker regresses the pupil area in a coarse-to-fine manner based on a revised Supervised Descent Method based eye-nose alignment. For sunglasses faces, because the eyes are occluded, our eye tracker uses whole face alignment with a revised Practical Facial Landmark Detector for pupil center tracking. Furthermore, we propose a structural inference-based re-weight network to predict eye position from non-occluded areas, such as the nose and mouth. The proposed re-weight sub-network revises the importance of different feature map positions and predicts the occluded eye positions by non-occluded parts. The proposed eye tracker is robust via a tracker-checker and a small model size. Experiments show that our method achieves high accuracy and speed, approximately 1.5 and 6.5 mm error for bare and sunglasses faces, respectively, at less than 10 ms on a 2.0GHz CPU. The evaluation dataset was captured indoors and outdoors to reflect multiple sunlight conditions. Our proposed method, combined with AR 3D HUDs, shows promising results for commercialization with low crosstalk 3D images. |
| first_indexed | 2024-03-08T14:27:23Z |
| format | Article |
| id | doaj.art-a55308b6cd0f498a98c48f2bd1dc1fbb |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T14:27:23Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a55308b6cd0f498a98c48f2bd1dc1fbb2024-01-13T00:01:16ZengIEEEIEEE Access2169-35362021-01-01912550812552210.1109/ACCESS.2021.31106449530401Real-Time Eye Tracking for Bare and Sunglasses-Wearing Faces for Augmented Reality 3D Head-Up DisplaysDongwoo Kang0https://orcid.org/0000-0001-7151-703XLin Ma1Department of Electronic and Electrical Engineering, Hongik University, Seoul, South KoreaSAIT China Lab, SRC-Beijing, Samsung Electronics, Beijing, ChinaEye pupil tracking is important for augmented reality (AR) three-dimensional (3D) head-up displays (HUDs). Accurate and fast eye tracking is still challenging due to multiple driving conditions with eye occlusions, such as wearing sunglasses. In this paper, we propose a system for commercial use that can handle practical driving conditions. Our system classifies human faces into bare faces and sunglasses faces, which are treated differently. For bare faces, our eye tracker regresses the pupil area in a coarse-to-fine manner based on a revised Supervised Descent Method based eye-nose alignment. For sunglasses faces, because the eyes are occluded, our eye tracker uses whole face alignment with a revised Practical Facial Landmark Detector for pupil center tracking. Furthermore, we propose a structural inference-based re-weight network to predict eye position from non-occluded areas, such as the nose and mouth. The proposed re-weight sub-network revises the importance of different feature map positions and predicts the occluded eye positions by non-occluded parts. The proposed eye tracker is robust via a tracker-checker and a small model size. Experiments show that our method achieves high accuracy and speed, approximately 1.5 and 6.5 mm error for bare and sunglasses faces, respectively, at less than 10 ms on a 2.0GHz CPU. The evaluation dataset was captured indoors and outdoors to reflect multiple sunlight conditions. Our proposed method, combined with AR 3D HUDs, shows promising results for commercialization with low crosstalk 3D images.https://ieeexplore.ieee.org/document/9530401/Eye trackingiris regressioneye position estimationaugmented reality (AR) displayautostereoscopic three-dimensional displayhead-up displays (HUDs) |
| spellingShingle | Dongwoo Kang Lin Ma Real-Time Eye Tracking for Bare and Sunglasses-Wearing Faces for Augmented Reality 3D Head-Up Displays IEEE Access Eye tracking iris regression eye position estimation augmented reality (AR) display autostereoscopic three-dimensional display head-up displays (HUDs) |
| title | Real-Time Eye Tracking for Bare and Sunglasses-Wearing Faces for Augmented Reality 3D Head-Up Displays |
| title_full | Real-Time Eye Tracking for Bare and Sunglasses-Wearing Faces for Augmented Reality 3D Head-Up Displays |
| title_fullStr | Real-Time Eye Tracking for Bare and Sunglasses-Wearing Faces for Augmented Reality 3D Head-Up Displays |
| title_full_unstemmed | Real-Time Eye Tracking for Bare and Sunglasses-Wearing Faces for Augmented Reality 3D Head-Up Displays |
| title_short | Real-Time Eye Tracking for Bare and Sunglasses-Wearing Faces for Augmented Reality 3D Head-Up Displays |
| title_sort | real time eye tracking for bare and sunglasses wearing faces for augmented reality 3d head up displays |
| topic | Eye tracking iris regression eye position estimation augmented reality (AR) display autostereoscopic three-dimensional display head-up displays (HUDs) |
| url | https://ieeexplore.ieee.org/document/9530401/ |
| work_keys_str_mv | AT dongwookang realtimeeyetrackingforbareandsunglasseswearingfacesforaugmentedreality3dheadupdisplays AT linma realtimeeyetrackingforbareandsunglasseswearingfacesforaugmentedreality3dheadupdisplays |