Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality
Realistic interactions with real objects (e.g., animals, toys, robots) in an augmented reality (AR) environment enhances the user experience. The common AR apps on the market achieve realistic interactions by superimposing pre-modeled virtual proxies on the real objects in the AR environment. This w...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/6/2763 |
| _version_ | 1827697314250096640 |
|---|---|
| author | Hanseob Kim Ghazanfar Ali Andréas Pastor Myungho Lee Gerard J. Kim Jae-In Hwang |
| author_facet | Hanseob Kim Ghazanfar Ali Andréas Pastor Myungho Lee Gerard J. Kim Jae-In Hwang |
| author_sort | Hanseob Kim |
| collection | DOAJ |
| description | Realistic interactions with real objects (e.g., animals, toys, robots) in an augmented reality (AR) environment enhances the user experience. The common AR apps on the market achieve realistic interactions by superimposing pre-modeled virtual proxies on the real objects in the AR environment. This way user perceives the interaction with virtual proxies as interaction with real objects. However, catering to environment change, shape deformation, and view update is not a trivial task. Our proposed method uses the dynamic silhouette of a real object to enable realistic interactions. Our approach is practical, lightweight, and requires no additional hardware besides the device camera. For a case study, we designed a mobile AR application to interact with real animal dolls. Our scenario included a virtual human performing four types of realistic interactions. Results demonstrated our method’s stability that does not require pre-modeled virtual proxies in case of shape deformation and view update. We also conducted a pilot study using our approach and reported significant improvements in user perception of spatial awareness and presence for realistic interactions with a virtual human. |
| first_indexed | 2024-03-10T13:06:21Z |
| format | Article |
| id | doaj.art-f9d07064c8a84c2c8f4e11574c86c7ac |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T13:06:21Z |
| publishDate | 2021-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-f9d07064c8a84c2c8f4e11574c86c7ac2023-11-21T11:09:27ZengMDPI AGApplied Sciences2076-34172021-03-01116276310.3390/app11062763Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented RealityHanseob Kim0Ghazanfar Ali1Andréas Pastor2Myungho Lee3Gerard J. Kim4Jae-In Hwang5Center for Artificial Intelligence, Korea Institute of Science and Technology (KIST), Seoul 02792, KoreaCenter for Artificial Intelligence, Korea Institute of Science and Technology (KIST), Seoul 02792, KoreaLS2N, University of Nantes, 44200 Nantes, FranceSchool of Computer Science and Engineering, Pusan National University, Busan 46241, KoreaDepartment of Computer Science and Engineering, Korea University, Seoul 02841, KoreaCenter for Artificial Intelligence, Korea Institute of Science and Technology (KIST), Seoul 02792, KoreaRealistic interactions with real objects (e.g., animals, toys, robots) in an augmented reality (AR) environment enhances the user experience. The common AR apps on the market achieve realistic interactions by superimposing pre-modeled virtual proxies on the real objects in the AR environment. This way user perceives the interaction with virtual proxies as interaction with real objects. However, catering to environment change, shape deformation, and view update is not a trivial task. Our proposed method uses the dynamic silhouette of a real object to enable realistic interactions. Our approach is practical, lightweight, and requires no additional hardware besides the device camera. For a case study, we designed a mobile AR application to interact with real animal dolls. Our scenario included a virtual human performing four types of realistic interactions. Results demonstrated our method’s stability that does not require pre-modeled virtual proxies in case of shape deformation and view update. We also conducted a pilot study using our approach and reported significant improvements in user perception of spatial awareness and presence for realistic interactions with a virtual human.https://www.mdpi.com/2076-3417/11/6/2763augmented realitymobile ARdeep learningsegmentationrealistic interactionvirtual human |
| spellingShingle | Hanseob Kim Ghazanfar Ali Andréas Pastor Myungho Lee Gerard J. Kim Jae-In Hwang Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality Applied Sciences augmented reality mobile AR deep learning segmentation realistic interaction virtual human |
| title | Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality |
| title_full | Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality |
| title_fullStr | Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality |
| title_full_unstemmed | Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality |
| title_short | Silhouettes from Real Objects Enable Realistic Interactions with a Virtual Human in Mobile Augmented Reality |
| title_sort | silhouettes from real objects enable realistic interactions with a virtual human in mobile augmented reality |
| topic | augmented reality mobile AR deep learning segmentation realistic interaction virtual human |
| url | https://www.mdpi.com/2076-3417/11/6/2763 |
| work_keys_str_mv | AT hanseobkim silhouettesfromrealobjectsenablerealisticinteractionswithavirtualhumaninmobileaugmentedreality AT ghazanfarali silhouettesfromrealobjectsenablerealisticinteractionswithavirtualhumaninmobileaugmentedreality AT andreaspastor silhouettesfromrealobjectsenablerealisticinteractionswithavirtualhumaninmobileaugmentedreality AT myungholee silhouettesfromrealobjectsenablerealisticinteractionswithavirtualhumaninmobileaugmentedreality AT gerardjkim silhouettesfromrealobjectsenablerealisticinteractionswithavirtualhumaninmobileaugmentedreality AT jaeinhwang silhouettesfromrealobjectsenablerealisticinteractionswithavirtualhumaninmobileaugmentedreality |