Online Hand Gesture Detection and Recognition for UAV Motion Planning
Recent advances in hand gesture recognition have produced more natural and intuitive methods of controlling unmanned aerial vehicles (UAVs). However, in unknown and cluttered environments, UAV motion planning requires the assistance of hand gesture interaction in complex flight tasks, which remains...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Machines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1702/11/2/210 |
| _version_ | 1797619733141389312 |
|---|---|
| author | Cong Lu Haoyang Zhang Yu Pei Liang Xie Ye Yan Erwei Yin Jing Jin |
| author_facet | Cong Lu Haoyang Zhang Yu Pei Liang Xie Ye Yan Erwei Yin Jing Jin |
| author_sort | Cong Lu |
| collection | DOAJ |
| description | Recent advances in hand gesture recognition have produced more natural and intuitive methods of controlling unmanned aerial vehicles (UAVs). However, in unknown and cluttered environments, UAV motion planning requires the assistance of hand gesture interaction in complex flight tasks, which remains a significant challenge. In this paper, a novel framework based on hand gesture interaction is proposed, to support efficient and robust UAV flight. A cascading structure, which includes Gaussian Native Bayes (GNB) and Random Forest (RF), was designed, to classify hand gestures based on the Six Degrees of Freedom (6DoF) inertial measurement units (IMUs) of the data glove. The hand gestures were mapped onto UAV’s flight commands, which corresponded to the direction of the UAV flight.The experimental results, which tested the 10 evaluated hand gestures, revealed the high accuracy of online hand gesture recognition under asynchronous detection (92%), and relatively low latency for interaction (average recognition time of 7.5 ms; average total time of 3 s).The average time of the UAV’s complex flight task was about 8 s shorter than that of the synchronous hand gesture detection and recognition. The proposed framework was validated as efficient and robust, with extensive benchmark comparisons in various complex real-world environments. |
| first_indexed | 2024-03-11T08:31:04Z |
| format | Article |
| id | doaj.art-050d1f9f7a52473e89b709d28f88070b |
| institution | Directory Open Access Journal |
| issn | 2075-1702 |
| language | English |
| last_indexed | 2024-03-11T08:31:04Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machines |
| spelling | doaj.art-050d1f9f7a52473e89b709d28f88070b2023-11-16T21:45:25ZengMDPI AGMachines2075-17022023-02-0111221010.3390/machines11020210Online Hand Gesture Detection and Recognition for UAV Motion PlanningCong Lu0Haoyang Zhang1Yu Pei2Liang Xie3Ye Yan4Erwei Yin5Jing Jin6School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, ChinaNational Innovation Institute of Defense Technology, Academy of Military Sciences, Beijing 100071, ChinaNational Innovation Institute of Defense Technology, Academy of Military Sciences, Beijing 100071, ChinaNational Innovation Institute of Defense Technology, Academy of Military Sciences, Beijing 100071, ChinaNational Innovation Institute of Defense Technology, Academy of Military Sciences, Beijing 100071, ChinaNational Innovation Institute of Defense Technology, Academy of Military Sciences, Beijing 100071, ChinaKey Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai 200237, ChinaRecent advances in hand gesture recognition have produced more natural and intuitive methods of controlling unmanned aerial vehicles (UAVs). However, in unknown and cluttered environments, UAV motion planning requires the assistance of hand gesture interaction in complex flight tasks, which remains a significant challenge. In this paper, a novel framework based on hand gesture interaction is proposed, to support efficient and robust UAV flight. A cascading structure, which includes Gaussian Native Bayes (GNB) and Random Forest (RF), was designed, to classify hand gestures based on the Six Degrees of Freedom (6DoF) inertial measurement units (IMUs) of the data glove. The hand gestures were mapped onto UAV’s flight commands, which corresponded to the direction of the UAV flight.The experimental results, which tested the 10 evaluated hand gestures, revealed the high accuracy of online hand gesture recognition under asynchronous detection (92%), and relatively low latency for interaction (average recognition time of 7.5 ms; average total time of 3 s).The average time of the UAV’s complex flight task was about 8 s shorter than that of the synchronous hand gesture detection and recognition. The proposed framework was validated as efficient and robust, with extensive benchmark comparisons in various complex real-world environments.https://www.mdpi.com/2075-1702/11/2/210IMU data glovehand gesture detectionhand gesture recognitionUAV motion planninginteraction efficiency |
| spellingShingle | Cong Lu Haoyang Zhang Yu Pei Liang Xie Ye Yan Erwei Yin Jing Jin Online Hand Gesture Detection and Recognition for UAV Motion Planning Machines IMU data glove hand gesture detection hand gesture recognition UAV motion planning interaction efficiency |
| title | Online Hand Gesture Detection and Recognition for UAV Motion Planning |
| title_full | Online Hand Gesture Detection and Recognition for UAV Motion Planning |
| title_fullStr | Online Hand Gesture Detection and Recognition for UAV Motion Planning |
| title_full_unstemmed | Online Hand Gesture Detection and Recognition for UAV Motion Planning |
| title_short | Online Hand Gesture Detection and Recognition for UAV Motion Planning |
| title_sort | online hand gesture detection and recognition for uav motion planning |
| topic | IMU data glove hand gesture detection hand gesture recognition UAV motion planning interaction efficiency |
| url | https://www.mdpi.com/2075-1702/11/2/210 |
| work_keys_str_mv | AT conglu onlinehandgesturedetectionandrecognitionforuavmotionplanning AT haoyangzhang onlinehandgesturedetectionandrecognitionforuavmotionplanning AT yupei onlinehandgesturedetectionandrecognitionforuavmotionplanning AT liangxie onlinehandgesturedetectionandrecognitionforuavmotionplanning AT yeyan onlinehandgesturedetectionandrecognitionforuavmotionplanning AT erweiyin onlinehandgesturedetectionandrecognitionforuavmotionplanning AT jingjin onlinehandgesturedetectionandrecognitionforuavmotionplanning |