DeepHandsVR: Hand Interface Using Deep Learning in Immersive Virtual Reality
This paper proposes a hand interface through a novel deep learning that provides easy and realistic interactions with hands in immersive virtual reality. The proposed interface is designed to provide a real-to-virtual direct hand interface using a controller to map a real hand gesture to a virtual h...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-11-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/9/11/1863 |
| _version_ | 1797548664131944448 |
|---|---|
| author | Taeseok Kang Minsu Chae Eunbin Seo Mingyu Kim Jinmo Kim |
| author_facet | Taeseok Kang Minsu Chae Eunbin Seo Mingyu Kim Jinmo Kim |
| author_sort | Taeseok Kang |
| collection | DOAJ |
| description | This paper proposes a hand interface through a novel deep learning that provides easy and realistic interactions with hands in immersive virtual reality. The proposed interface is designed to provide a real-to-virtual direct hand interface using a controller to map a real hand gesture to a virtual hand in an easy and simple structure. In addition, a gesture-to-action interface that expresses the process of gesture to action in real-time without the necessity of a graphical user interface (GUI) used in existing interactive applications is proposed. This interface uses the method of applying image classification training process of capturing a 3D virtual hand gesture model as a 2D image using a deep learning model, convolutional neural network (CNN). The key objective of this process is to provide users with intuitive and realistic interactions that feature convenient operation in immersive virtual reality. To achieve this, an application that can compare and analyze the proposed interface and the existing GUI was developed. Next, a survey experiment was conducted to statistically analyze and evaluate the positive effects on the sense of presence through user satisfaction with the interface experience. |
| first_indexed | 2024-03-10T15:02:52Z |
| format | Article |
| id | doaj.art-2f66bdf959e24c8ea1d444fc6ad35c1a |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-10T15:02:52Z |
| publishDate | 2020-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-2f66bdf959e24c8ea1d444fc6ad35c1a2023-11-20T20:01:26ZengMDPI AGElectronics2079-92922020-11-01911186310.3390/electronics9111863DeepHandsVR: Hand Interface Using Deep Learning in Immersive Virtual RealityTaeseok Kang0Minsu Chae1Eunbin Seo2Mingyu Kim3Jinmo Kim4Division of Computer Engineering, Hansung University, Seoul 02876, KoreaDivision of Computer Engineering, Hansung University, Seoul 02876, KoreaDivision of Computer Engineering, Hansung University, Seoul 02876, KoreaProgram in Visual Information Processing, Korea University, Seoul 02841, KoreaDivision of Computer Engineering, Hansung University, Seoul 02876, KoreaThis paper proposes a hand interface through a novel deep learning that provides easy and realistic interactions with hands in immersive virtual reality. The proposed interface is designed to provide a real-to-virtual direct hand interface using a controller to map a real hand gesture to a virtual hand in an easy and simple structure. In addition, a gesture-to-action interface that expresses the process of gesture to action in real-time without the necessity of a graphical user interface (GUI) used in existing interactive applications is proposed. This interface uses the method of applying image classification training process of capturing a 3D virtual hand gesture model as a 2D image using a deep learning model, convolutional neural network (CNN). The key objective of this process is to provide users with intuitive and realistic interactions that feature convenient operation in immersive virtual reality. To achieve this, an application that can compare and analyze the proposed interface and the existing GUI was developed. Next, a survey experiment was conducted to statistically analyze and evaluate the positive effects on the sense of presence through user satisfaction with the interface experience.https://www.mdpi.com/2079-9292/9/11/1863hand interfaceimmersive virtual realitydeep learninginteractionpresence |
| spellingShingle | Taeseok Kang Minsu Chae Eunbin Seo Mingyu Kim Jinmo Kim DeepHandsVR: Hand Interface Using Deep Learning in Immersive Virtual Reality Electronics hand interface immersive virtual reality deep learning interaction presence |
| title | DeepHandsVR: Hand Interface Using Deep Learning in Immersive Virtual Reality |
| title_full | DeepHandsVR: Hand Interface Using Deep Learning in Immersive Virtual Reality |
| title_fullStr | DeepHandsVR: Hand Interface Using Deep Learning in Immersive Virtual Reality |
| title_full_unstemmed | DeepHandsVR: Hand Interface Using Deep Learning in Immersive Virtual Reality |
| title_short | DeepHandsVR: Hand Interface Using Deep Learning in Immersive Virtual Reality |
| title_sort | deephandsvr hand interface using deep learning in immersive virtual reality |
| topic | hand interface immersive virtual reality deep learning interaction presence |
| url | https://www.mdpi.com/2079-9292/9/11/1863 |
| work_keys_str_mv | AT taeseokkang deephandsvrhandinterfaceusingdeeplearninginimmersivevirtualreality AT minsuchae deephandsvrhandinterfaceusingdeeplearninginimmersivevirtualreality AT eunbinseo deephandsvrhandinterfaceusingdeeplearninginimmersivevirtualreality AT mingyukim deephandsvrhandinterfaceusingdeeplearninginimmersivevirtualreality AT jinmokim deephandsvrhandinterfaceusingdeeplearninginimmersivevirtualreality |