3D Point-of-Intention Determination Using a Multimodal Fusion of Hand Pointing and Eye Gaze for a 3D Display
This paper proposes a three-dimensional (3D) point-of-intention (POI) determination method using multimodal fusion between hand pointing and eye gaze for a 3D virtual display. In the method, the finger joint forms of the pointing hand sensed by a Leap Motion sensor are first detected as pointing int...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/4/1155 |
| _version_ | 1797413478231703552 |
|---|---|
| author | Suparat Yeamkuan Kosin Chamnongthai |
| author_facet | Suparat Yeamkuan Kosin Chamnongthai |
| author_sort | Suparat Yeamkuan |
| collection | DOAJ |
| description | This paper proposes a three-dimensional (3D) point-of-intention (POI) determination method using multimodal fusion between hand pointing and eye gaze for a 3D virtual display. In the method, the finger joint forms of the pointing hand sensed by a Leap Motion sensor are first detected as pointing intention candidates. Subsequently, differences with neighboring frames, which should be during hand pointing period, are checked by AND logic with the hand-pointing intention candidates. A crossing point between the eye gaze and hand pointing lines is finally decided by the closest distance concept. In order to evaluate the performance of the proposed method, experiments with ten participants, in which they looked at and pointed at nine test points for approximately five second each, were performed. The experimental results show the proposed method measures 3D POIs at 75 cm, 85 cm, and 95 cm with average distance errors of 4.67%, 5.38%, and 5.71%, respectively. |
| first_indexed | 2024-03-09T05:18:31Z |
| format | Article |
| id | doaj.art-c3746c5f00844802a5edf8744a63502f |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T05:18:31Z |
| publishDate | 2021-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-c3746c5f00844802a5edf8744a63502f2023-12-03T12:43:18ZengMDPI AGSensors1424-82202021-02-01214115510.3390/s210411553D Point-of-Intention Determination Using a Multimodal Fusion of Hand Pointing and Eye Gaze for a 3D DisplaySuparat Yeamkuan0Kosin Chamnongthai1Department of Electronic and Telecommunication Engineering, Faculty of Engineering, King Mongkut’s University of Technology Thonburi, Bangkok 10140, ThailandDepartment of Electronic and Telecommunication Engineering, Faculty of Engineering, King Mongkut’s University of Technology Thonburi, Bangkok 10140, ThailandThis paper proposes a three-dimensional (3D) point-of-intention (POI) determination method using multimodal fusion between hand pointing and eye gaze for a 3D virtual display. In the method, the finger joint forms of the pointing hand sensed by a Leap Motion sensor are first detected as pointing intention candidates. Subsequently, differences with neighboring frames, which should be during hand pointing period, are checked by AND logic with the hand-pointing intention candidates. A crossing point between the eye gaze and hand pointing lines is finally decided by the closest distance concept. In order to evaluate the performance of the proposed method, experiments with ten participants, in which they looked at and pointed at nine test points for approximately five second each, were performed. The experimental results show the proposed method measures 3D POIs at 75 cm, 85 cm, and 95 cm with average distance errors of 4.67%, 5.38%, and 5.71%, respectively.https://www.mdpi.com/1424-8220/21/4/1155multimodal systemssensor fusioneye trackinghand trackinghand recognition |
| spellingShingle | Suparat Yeamkuan Kosin Chamnongthai 3D Point-of-Intention Determination Using a Multimodal Fusion of Hand Pointing and Eye Gaze for a 3D Display Sensors multimodal systems sensor fusion eye tracking hand tracking hand recognition |
| title | 3D Point-of-Intention Determination Using a Multimodal Fusion of Hand Pointing and Eye Gaze for a 3D Display |
| title_full | 3D Point-of-Intention Determination Using a Multimodal Fusion of Hand Pointing and Eye Gaze for a 3D Display |
| title_fullStr | 3D Point-of-Intention Determination Using a Multimodal Fusion of Hand Pointing and Eye Gaze for a 3D Display |
| title_full_unstemmed | 3D Point-of-Intention Determination Using a Multimodal Fusion of Hand Pointing and Eye Gaze for a 3D Display |
| title_short | 3D Point-of-Intention Determination Using a Multimodal Fusion of Hand Pointing and Eye Gaze for a 3D Display |
| title_sort | 3d point of intention determination using a multimodal fusion of hand pointing and eye gaze for a 3d display |
| topic | multimodal systems sensor fusion eye tracking hand tracking hand recognition |
| url | https://www.mdpi.com/1424-8220/21/4/1155 |
| work_keys_str_mv | AT suparatyeamkuan 3dpointofintentiondeterminationusingamultimodalfusionofhandpointingandeyegazefora3ddisplay AT kosinchamnongthai 3dpointofintentiondeterminationusingamultimodalfusionofhandpointingandeyegazefora3ddisplay |