Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approach
The recent fast development in computer vision and mobile sensor technology such as mobile LiDAR and RGB-D cameras is pushing the boundary of the technology to suit the need of real-life applications in the fields of Augmented Reality (AR), robotics, indoor GIS and self-driving. Camera localization...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2020-06-01
|
| Series: | International Journal of Digital Earth |
| Subjects: | |
| Online Access: | http://dx.doi.org/10.1080/17538947.2018.1564379 |
| _version_ | 1797678543014985728 |
|---|---|
| author | Jing Li Chengyi Wang Xuejie Kang Qiang Zhao |
| author_facet | Jing Li Chengyi Wang Xuejie Kang Qiang Zhao |
| author_sort | Jing Li |
| collection | DOAJ |
| description | The recent fast development in computer vision and mobile sensor technology such as mobile LiDAR and RGB-D cameras is pushing the boundary of the technology to suit the need of real-life applications in the fields of Augmented Reality (AR), robotics, indoor GIS and self-driving. Camera localization is often a key and enabling technology among these applications. In this paper, we developed a novel camera localization workflow based on a highly accurate 3D prior map optimized by our RGB-D SLAM method in conjunction with a deep learning routine trained using consecutive video frames labeled with high precision camera pose. Furthermore, an AR registration method tightly coupled with a game engine is proposed, which incorporates the proposed localization algorithm and aligns the real Kinetic camera with a virtual camera of the game engine to facilitate AR application development in an integrated manner. The experimental results show that the localization accuracy can achieve an average error of 35 cm based on a fine-tuned prior 3D feature database at 3 cm accuracy compared against the ground-truth 3D LiDAR map. The influence of the localization accuracy on the visual effect of AR overlay is also demonstrated and the alignment of the real and virtual camera streamlines the implementation of AR fire emergency response demo in a Virtual Geographic Environment. |
| first_indexed | 2024-03-11T23:01:22Z |
| format | Article |
| id | doaj.art-37ebcb98a5094c589889289b2b8886fc |
| institution | Directory Open Access Journal |
| issn | 1753-8947 1753-8955 |
| language | English |
| last_indexed | 2024-03-11T23:01:22Z |
| publishDate | 2020-06-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | International Journal of Digital Earth |
| spelling | doaj.art-37ebcb98a5094c589889289b2b8886fc2023-09-21T14:57:08ZengTaylor & Francis GroupInternational Journal of Digital Earth1753-89471753-89552020-06-0113672774110.1080/17538947.2018.15643791564379Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approachJing Li0Chengyi Wang1Xuejie Kang2Qiang Zhao3Institute of Remote Sensing and Digital Earth, Chinese Academy of SciencesInstitute of Remote Sensing and Digital Earth, Chinese Academy of SciencesInstitute of Remote Sensing and Digital Earth, Chinese Academy of SciencesInstitute of Remote Sensing and Digital Earth, Chinese Academy of SciencesThe recent fast development in computer vision and mobile sensor technology such as mobile LiDAR and RGB-D cameras is pushing the boundary of the technology to suit the need of real-life applications in the fields of Augmented Reality (AR), robotics, indoor GIS and self-driving. Camera localization is often a key and enabling technology among these applications. In this paper, we developed a novel camera localization workflow based on a highly accurate 3D prior map optimized by our RGB-D SLAM method in conjunction with a deep learning routine trained using consecutive video frames labeled with high precision camera pose. Furthermore, an AR registration method tightly coupled with a game engine is proposed, which incorporates the proposed localization algorithm and aligns the real Kinetic camera with a virtual camera of the game engine to facilitate AR application development in an integrated manner. The experimental results show that the localization accuracy can achieve an average error of 35 cm based on a fine-tuned prior 3D feature database at 3 cm accuracy compared against the ground-truth 3D LiDAR map. The influence of the localization accuracy on the visual effect of AR overlay is also demonstrated and the alignment of the real and virtual camera streamlines the implementation of AR fire emergency response demo in a Virtual Geographic Environment.http://dx.doi.org/10.1080/17538947.2018.1564379augmented reality (ar)virtual geographic environment (vge)indoor positioningindoor gislidar3d reconstructionlocalizationcamera pose |
| spellingShingle | Jing Li Chengyi Wang Xuejie Kang Qiang Zhao Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approach International Journal of Digital Earth augmented reality (ar) virtual geographic environment (vge) indoor positioning indoor gis lidar 3d reconstruction localization camera pose |
| title | Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approach |
| title_full | Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approach |
| title_fullStr | Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approach |
| title_full_unstemmed | Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approach |
| title_short | Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approach |
| title_sort | camera localization for augmented reality and indoor positioning a vision based 3d feature database approach |
| topic | augmented reality (ar) virtual geographic environment (vge) indoor positioning indoor gis lidar 3d reconstruction localization camera pose |
| url | http://dx.doi.org/10.1080/17538947.2018.1564379 |
| work_keys_str_mv | AT jingli cameralocalizationforaugmentedrealityandindoorpositioningavisionbased3dfeaturedatabaseapproach AT chengyiwang cameralocalizationforaugmentedrealityandindoorpositioningavisionbased3dfeaturedatabaseapproach AT xuejiekang cameralocalizationforaugmentedrealityandindoorpositioningavisionbased3dfeaturedatabaseapproach AT qiangzhao cameralocalizationforaugmentedrealityandindoorpositioningavisionbased3dfeaturedatabaseapproach |