Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene Understanding
We propose an intelligent human−unmanned aerial vehicle (UAV) interaction system, in which, instead of using the conventional remote controller, the UAV flight actions are controlled by a deep learning-based action−gesture joint detection system. The Resnet-based scene-understand...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-08-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/9/16/3277 |
| _version_ | 1817988424535113728 |
|---|---|
| author | Bo Chen Chunsheng Hua Decai Li Yuqing He Jianda Han |
| author_facet | Bo Chen Chunsheng Hua Decai Li Yuqing He Jianda Han |
| author_sort | Bo Chen |
| collection | DOAJ |
| description | We propose an intelligent human−unmanned aerial vehicle (UAV) interaction system, in which, instead of using the conventional remote controller, the UAV flight actions are controlled by a deep learning-based action−gesture joint detection system. The Resnet-based scene-understanding algorithm is introduced into the proposed system to enable the UAV to adjust its flight strategy automatically, according to the flying conditions. Meanwhile, both the deep learning-based action detection and multi-feature cascade gesture recognition methods are employed by a cross-validation process to create the corresponding flight action. The effectiveness and efficiency of the proposed system are confirmed by its application to controlling the flight action of a real flying UAV for more than 3 h. |
| first_indexed | 2024-04-14T00:34:03Z |
| format | Article |
| id | doaj.art-726d3f6061754c9b8f995dea76cce4d8 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-04-14T00:34:03Z |
| publishDate | 2019-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-726d3f6061754c9b8f995dea76cce4d82022-12-22T02:22:27ZengMDPI AGApplied Sciences2076-34172019-08-01916327710.3390/app9163277app9163277Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene UnderstandingBo Chen0Chunsheng Hua1Decai Li2Yuqing He3Jianda Han4State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110000, ChinaCollege of Information, Liaoning University, Shenyang 110000, ChinaState Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110000, ChinaState Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110000, ChinaInstitute of Robotics and Automatic Information System, College of Artificial Intelligence, Nankai University, Tianjin 300000, ChinaWe propose an intelligent human−unmanned aerial vehicle (UAV) interaction system, in which, instead of using the conventional remote controller, the UAV flight actions are controlled by a deep learning-based action−gesture joint detection system. The Resnet-based scene-understanding algorithm is introduced into the proposed system to enable the UAV to adjust its flight strategy automatically, according to the flying conditions. Meanwhile, both the deep learning-based action detection and multi-feature cascade gesture recognition methods are employed by a cross-validation process to create the corresponding flight action. The effectiveness and efficiency of the proposed system are confirmed by its application to controlling the flight action of a real flying UAV for more than 3 h.https://www.mdpi.com/2076-3417/9/16/3277action detectiongesture recognitionscene understandingjoint cross validation |
| spellingShingle | Bo Chen Chunsheng Hua Decai Li Yuqing He Jianda Han Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene Understanding Applied Sciences action detection gesture recognition scene understanding joint cross validation |
| title | Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene Understanding |
| title_full | Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene Understanding |
| title_fullStr | Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene Understanding |
| title_full_unstemmed | Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene Understanding |
| title_short | Intelligent Human–UAV Interaction System with Joint Cross-Validation over Action–Gesture Recognition and Scene Understanding |
| title_sort | intelligent human uav interaction system with joint cross validation over action gesture recognition and scene understanding |
| topic | action detection gesture recognition scene understanding joint cross validation |
| url | https://www.mdpi.com/2076-3417/9/16/3277 |
| work_keys_str_mv | AT bochen intelligenthumanuavinteractionsystemwithjointcrossvalidationoveractiongesturerecognitionandsceneunderstanding AT chunshenghua intelligenthumanuavinteractionsystemwithjointcrossvalidationoveractiongesturerecognitionandsceneunderstanding AT decaili intelligenthumanuavinteractionsystemwithjointcrossvalidationoveractiongesturerecognitionandsceneunderstanding AT yuqinghe intelligenthumanuavinteractionsystemwithjointcrossvalidationoveractiongesturerecognitionandsceneunderstanding AT jiandahan intelligenthumanuavinteractionsystemwithjointcrossvalidationoveractiongesturerecognitionandsceneunderstanding |