From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation
In this paper, we propose two novel AR glasses pose estimation algorithms from single infrared images by using 3D point clouds as an intermediate representation. Our first approach “PointsToRotation” is based on a Deep Neural Network alone, whereas our second approach “PointsToPose” is a hybrid mode...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-04-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/7/5/80 |
| _version_ | 1797536113411227648 |
|---|---|
| author | Ahmet Firintepe Carolin Vey Stylianos Asteriadis Alain Pagani Didier Stricker |
| author_facet | Ahmet Firintepe Carolin Vey Stylianos Asteriadis Alain Pagani Didier Stricker |
| author_sort | Ahmet Firintepe |
| collection | DOAJ |
| description | In this paper, we propose two novel AR glasses pose estimation algorithms from single infrared images by using 3D point clouds as an intermediate representation. Our first approach “PointsToRotation” is based on a Deep Neural Network alone, whereas our second approach “PointsToPose” is a hybrid model combining Deep Learning and a voting-based mechanism. Our methods utilize a point cloud estimator, which we trained on multi-view infrared images in a semi-supervised manner, generating point clouds based on one image only. We generate a point cloud dataset with our point cloud estimator using the HMDPose dataset, consisting of multi-view infrared images of various AR glasses with the corresponding 6-DoF poses. In comparison to another point cloud-based 6-DoF pose estimation named CloudPose, we achieve an error reduction of around 50%. Compared to a state-of-the-art image-based method, we reduce the pose estimation error by around 96%. |
| first_indexed | 2024-03-10T11:55:02Z |
| format | Article |
| id | doaj.art-6bab9d9b13654047bdbc0e83431116aa |
| institution | Directory Open Access Journal |
| issn | 2313-433X |
| language | English |
| last_indexed | 2024-03-10T11:55:02Z |
| publishDate | 2021-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj.art-6bab9d9b13654047bdbc0e83431116aa2023-11-21T17:21:45ZengMDPI AGJournal of Imaging2313-433X2021-04-01758010.3390/jimaging7050080From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose EstimationAhmet Firintepe0Carolin Vey1Stylianos Asteriadis2Alain Pagani3Didier Stricker4BMW Group Research, New Technologies, Innovations, 85748 Munich, GermanyBMW Group Research, New Technologies, Innovations, 85748 Munich, GermanyDepartment of Data Science and Knowledge Engineering, Maastricht University, 6211 TE Maastricht, The NetherlandsGerman Research Center for Artificial Intelligence (DFKI), 67653 Kaiserslautern, GermanyDepartment of Informatics, University of Kaiserslautern, 67653 Kaiserslautern, GermanyIn this paper, we propose two novel AR glasses pose estimation algorithms from single infrared images by using 3D point clouds as an intermediate representation. Our first approach “PointsToRotation” is based on a Deep Neural Network alone, whereas our second approach “PointsToPose” is a hybrid model combining Deep Learning and a voting-based mechanism. Our methods utilize a point cloud estimator, which we trained on multi-view infrared images in a semi-supervised manner, generating point clouds based on one image only. We generate a point cloud dataset with our point cloud estimator using the HMDPose dataset, consisting of multi-view infrared images of various AR glasses with the corresponding 6-DoF poses. In comparison to another point cloud-based 6-DoF pose estimation named CloudPose, we achieve an error reduction of around 50%. Compared to a state-of-the-art image-based method, we reduce the pose estimation error by around 96%.https://www.mdpi.com/2313-433X/7/5/80computer visionaugmented realityobject pose estimationpoint cloudsdeep learning |
| spellingShingle | Ahmet Firintepe Carolin Vey Stylianos Asteriadis Alain Pagani Didier Stricker From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation Journal of Imaging computer vision augmented reality object pose estimation point clouds deep learning |
| title | From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation |
| title_full | From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation |
| title_fullStr | From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation |
| title_full_unstemmed | From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation |
| title_short | From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation |
| title_sort | from ir images to point clouds to pose point cloud based ar glasses pose estimation |
| topic | computer vision augmented reality object pose estimation point clouds deep learning |
| url | https://www.mdpi.com/2313-433X/7/5/80 |
| work_keys_str_mv | AT ahmetfirintepe fromirimagestopointcloudstoposepointcloudbasedarglassesposeestimation AT carolinvey fromirimagestopointcloudstoposepointcloudbasedarglassesposeestimation AT stylianosasteriadis fromirimagestopointcloudstoposepointcloudbasedarglassesposeestimation AT alainpagani fromirimagestopointcloudstoposepointcloudbasedarglassesposeestimation AT didierstricker fromirimagestopointcloudstoposepointcloudbasedarglassesposeestimation |