Affine Layer-Enabled Transfer Learning for Eye Tracking with Facial Feature Detection in Human–Machine Interactions
Eye tracking is an important technique for realizing safe and efficient human–machine interaction. This study proposes a facial-based eye tracking system that only relies on a non-intrusive, low-cost web camera by leveraging a data-driven approach. To address the challenge of rapid deployment to a n...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-09-01
|
| Series: | Machines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1702/10/10/853 |
| _version_ | 1827597577381478400 |
|---|---|
| author | Zhongxu Hu Yiran Zhang Chen Lv |
| author_facet | Zhongxu Hu Yiran Zhang Chen Lv |
| author_sort | Zhongxu Hu |
| collection | DOAJ |
| description | Eye tracking is an important technique for realizing safe and efficient human–machine interaction. This study proposes a facial-based eye tracking system that only relies on a non-intrusive, low-cost web camera by leveraging a data-driven approach. To address the challenge of rapid deployment to a new scenario and reduce the workload of the data collection, this study proposes an efficient transfer learning approach that includes a novel affine layer to bridge the gap between the source domain and the target domain to improve the transfer learning performance. Furthermore, a calibration technique is also introduced in this study for model performance optimization. To verify the proposed approach, a series of comparative experiments are conducted on a designed experimental platform to evaluate the effects of various transfer learning strategies, the proposed affine layer module, and the calibration technique. The experiment results showed that the proposed affine layer can improve the model’s performance by <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>7</mn><mo>%</mo></mrow></semantics></math></inline-formula> (without calibration) and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>4</mn><mo>%</mo></mrow></semantics></math></inline-formula> (with calibration), and the proposed approach can achieve state-of-the-art performance when compared to the others. |
| first_indexed | 2024-03-09T03:34:26Z |
| format | Article |
| id | doaj.art-3f7aaedcc421473baffb1a5a3cb78bf8 |
| institution | Directory Open Access Journal |
| issn | 2075-1702 |
| language | English |
| last_indexed | 2024-03-09T03:34:26Z |
| publishDate | 2022-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machines |
| spelling | doaj.art-3f7aaedcc421473baffb1a5a3cb78bf82023-12-03T14:50:46ZengMDPI AGMachines2075-17022022-09-01101085310.3390/machines10100853Affine Layer-Enabled Transfer Learning for Eye Tracking with Facial Feature Detection in Human–Machine InteractionsZhongxu Hu0Yiran Zhang1Chen Lv2School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 637459, SingaporeSchool of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 637459, SingaporeSchool of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 637459, SingaporeEye tracking is an important technique for realizing safe and efficient human–machine interaction. This study proposes a facial-based eye tracking system that only relies on a non-intrusive, low-cost web camera by leveraging a data-driven approach. To address the challenge of rapid deployment to a new scenario and reduce the workload of the data collection, this study proposes an efficient transfer learning approach that includes a novel affine layer to bridge the gap between the source domain and the target domain to improve the transfer learning performance. Furthermore, a calibration technique is also introduced in this study for model performance optimization. To verify the proposed approach, a series of comparative experiments are conducted on a designed experimental platform to evaluate the effects of various transfer learning strategies, the proposed affine layer module, and the calibration technique. The experiment results showed that the proposed affine layer can improve the model’s performance by <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>7</mn><mo>%</mo></mrow></semantics></math></inline-formula> (without calibration) and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>4</mn><mo>%</mo></mrow></semantics></math></inline-formula> (with calibration), and the proposed approach can achieve state-of-the-art performance when compared to the others.https://www.mdpi.com/2075-1702/10/10/853human–machine interactionseye trackingaffine layertransfer learningfacial featurevision system |
| spellingShingle | Zhongxu Hu Yiran Zhang Chen Lv Affine Layer-Enabled Transfer Learning for Eye Tracking with Facial Feature Detection in Human–Machine Interactions Machines human–machine interactions eye tracking affine layer transfer learning facial feature vision system |
| title | Affine Layer-Enabled Transfer Learning for Eye Tracking with Facial Feature Detection in Human–Machine Interactions |
| title_full | Affine Layer-Enabled Transfer Learning for Eye Tracking with Facial Feature Detection in Human–Machine Interactions |
| title_fullStr | Affine Layer-Enabled Transfer Learning for Eye Tracking with Facial Feature Detection in Human–Machine Interactions |
| title_full_unstemmed | Affine Layer-Enabled Transfer Learning for Eye Tracking with Facial Feature Detection in Human–Machine Interactions |
| title_short | Affine Layer-Enabled Transfer Learning for Eye Tracking with Facial Feature Detection in Human–Machine Interactions |
| title_sort | affine layer enabled transfer learning for eye tracking with facial feature detection in human machine interactions |
| topic | human–machine interactions eye tracking affine layer transfer learning facial feature vision system |
| url | https://www.mdpi.com/2075-1702/10/10/853 |
| work_keys_str_mv | AT zhongxuhu affinelayerenabledtransferlearningforeyetrackingwithfacialfeaturedetectioninhumanmachineinteractions AT yiranzhang affinelayerenabledtransferlearningforeyetrackingwithfacialfeaturedetectioninhumanmachineinteractions AT chenlv affinelayerenabledtransferlearningforeyetrackingwithfacialfeaturedetectioninhumanmachineinteractions |