Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device
Mixed reality technology can give humans an intuitive visual experience, and combined with the multi-source information of the human body, it can provide a comfortable human–robot interaction experience. This paper applies a mixed reality device (Hololens2) to provide interactive communication betwe...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-10-01
|
| Series: | Biomimetics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-7673/8/6/479 |
| _version_ | 1797574577060052992 |
|---|---|
| author | Hongwei Jing Tianjiao Zheng Qinghua Zhang Kerui Sun Lele Li Mingzhu Lai Jie Zhao Yanhe Zhu |
| author_facet | Hongwei Jing Tianjiao Zheng Qinghua Zhang Kerui Sun Lele Li Mingzhu Lai Jie Zhao Yanhe Zhu |
| author_sort | Hongwei Jing |
| collection | DOAJ |
| description | Mixed reality technology can give humans an intuitive visual experience, and combined with the multi-source information of the human body, it can provide a comfortable human–robot interaction experience. This paper applies a mixed reality device (Hololens2) to provide interactive communication between the wearer and the wearable robotic limb (supernumerary robotic limb, SRL). Hololens2 can obtain human body information, including eye gaze, hand gestures, voice input, etc. It can also provide feedback information to the wearer through augmented reality and audio output, which is the communication bridge needed in human–robot interaction. Implementing a wearable robotic arm integrated with HoloLens2 is proposed to augment the wearer’s capabilities. Taking two typical practical tasks of cable installation and electrical connector soldering in aircraft manufacturing as examples, the task models and interaction scheme are designed. Finally, human augmentation is evaluated in terms of task completion time statistics. |
| first_indexed | 2024-03-10T21:24:27Z |
| format | Article |
| id | doaj.art-60736b5291bf4e36a01d5f4728fd5f68 |
| institution | Directory Open Access Journal |
| issn | 2313-7673 |
| language | English |
| last_indexed | 2024-03-10T21:24:27Z |
| publishDate | 2023-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Biomimetics |
| spelling | doaj.art-60736b5291bf4e36a01d5f4728fd5f682023-11-19T15:48:44ZengMDPI AGBiomimetics2313-76732023-10-018647910.3390/biomimetics8060479Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality DeviceHongwei Jing0Tianjiao Zheng1Qinghua Zhang2Kerui Sun3Lele Li4Mingzhu Lai5Jie Zhao6Yanhe Zhu7State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, ChinaState Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, ChinaState Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, ChinaState Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, ChinaState Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, ChinaSchool of Mathematics and Statistics, Hainan Normal University, Haikou 571158, ChinaState Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, ChinaState Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, ChinaMixed reality technology can give humans an intuitive visual experience, and combined with the multi-source information of the human body, it can provide a comfortable human–robot interaction experience. This paper applies a mixed reality device (Hololens2) to provide interactive communication between the wearer and the wearable robotic limb (supernumerary robotic limb, SRL). Hololens2 can obtain human body information, including eye gaze, hand gestures, voice input, etc. It can also provide feedback information to the wearer through augmented reality and audio output, which is the communication bridge needed in human–robot interaction. Implementing a wearable robotic arm integrated with HoloLens2 is proposed to augment the wearer’s capabilities. Taking two typical practical tasks of cable installation and electrical connector soldering in aircraft manufacturing as examples, the task models and interaction scheme are designed. Finally, human augmentation is evaluated in terms of task completion time statistics.https://www.mdpi.com/2313-7673/8/6/479human augmentationwearable robotic limbsupernumerary robotic limbmixed realityhuman–robot interaction |
| spellingShingle | Hongwei Jing Tianjiao Zheng Qinghua Zhang Kerui Sun Lele Li Mingzhu Lai Jie Zhao Yanhe Zhu Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device Biomimetics human augmentation wearable robotic limb supernumerary robotic limb mixed reality human–robot interaction |
| title | Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device |
| title_full | Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device |
| title_fullStr | Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device |
| title_full_unstemmed | Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device |
| title_short | Human Operation Augmentation through Wearable Robotic Limb Integrated with Mixed Reality Device |
| title_sort | human operation augmentation through wearable robotic limb integrated with mixed reality device |
| topic | human augmentation wearable robotic limb supernumerary robotic limb mixed reality human–robot interaction |
| url | https://www.mdpi.com/2313-7673/8/6/479 |
| work_keys_str_mv | AT hongweijing humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice AT tianjiaozheng humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice AT qinghuazhang humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice AT keruisun humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice AT leleli humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice AT mingzhulai humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice AT jiezhao humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice AT yanhezhu humanoperationaugmentationthroughwearableroboticlimbintegratedwithmixedrealitydevice |