Regression-Based Camera Pose Estimation through Multi-Level Local Features and Global Features
Accurate and robust camera pose estimation is essential for high-level applications such as augmented reality and autonomous driving. Despite the development of global feature-based camera pose regression methods and local feature-based matching guided pose estimation methods, challenging conditions...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/8/4063 |
| _version_ | 1797603504557129728 |
|---|---|
| author | Meng Xu Zhihuang Zhang Yuanhao Gong Stefan Poslad |
| author_facet | Meng Xu Zhihuang Zhang Yuanhao Gong Stefan Poslad |
| author_sort | Meng Xu |
| collection | DOAJ |
| description | Accurate and robust camera pose estimation is essential for high-level applications such as augmented reality and autonomous driving. Despite the development of global feature-based camera pose regression methods and local feature-based matching guided pose estimation methods, challenging conditions, such as illumination changes and viewpoint changes, as well as inaccurate keypoint localization, continue to affect the performance of camera pose estimation. In this paper, we propose a novel relative camera pose regression framework that uses global features with rotation consistency and local features with rotation invariance. First, we apply a multi-level deformable network to detect and describe local features, which can learn appearances and gradient information sensitive to rotation variants. Second, we process the detection and description processes using the results from pixel correspondences of the input image pairs. Finally, we propose a novel loss that combines relative regression loss and absolute regression loss, incorporating global features with geometric constraints to optimize the pose estimation model. Our extensive experiments report satisfactory accuracy on the 7Scenes dataset with an average mean translation error of 0.18 m and a rotation error of 7.44° using image pairs as input. Ablation studies were also conducted to verify the effectiveness of the proposed method in the tasks of pose estimation and image matching using the 7Scenes and HPatches datasets. |
| first_indexed | 2024-03-11T04:33:04Z |
| format | Article |
| id | doaj.art-799a8f8992864b6785cf8c4fc99d473c |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T04:33:04Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-799a8f8992864b6785cf8c4fc99d473c2023-11-17T21:18:32ZengMDPI AGSensors1424-82202023-04-01238406310.3390/s23084063Regression-Based Camera Pose Estimation through Multi-Level Local Features and Global FeaturesMeng Xu0Zhihuang Zhang1Yuanhao Gong2Stefan Poslad3School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UKSchool of Vehicle and Mobility, Tsinghua University, Beijing 100084, ChinaCollege of Electronics and Information Engineering, Shenzhen University, Shenzhen 518061, ChinaSchool of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UKAccurate and robust camera pose estimation is essential for high-level applications such as augmented reality and autonomous driving. Despite the development of global feature-based camera pose regression methods and local feature-based matching guided pose estimation methods, challenging conditions, such as illumination changes and viewpoint changes, as well as inaccurate keypoint localization, continue to affect the performance of camera pose estimation. In this paper, we propose a novel relative camera pose regression framework that uses global features with rotation consistency and local features with rotation invariance. First, we apply a multi-level deformable network to detect and describe local features, which can learn appearances and gradient information sensitive to rotation variants. Second, we process the detection and description processes using the results from pixel correspondences of the input image pairs. Finally, we propose a novel loss that combines relative regression loss and absolute regression loss, incorporating global features with geometric constraints to optimize the pose estimation model. Our extensive experiments report satisfactory accuracy on the 7Scenes dataset with an average mean translation error of 0.18 m and a rotation error of 7.44° using image pairs as input. Ablation studies were also conducted to verify the effectiveness of the proposed method in the tasks of pose estimation and image matching using the 7Scenes and HPatches datasets.https://www.mdpi.com/1424-8220/23/8/4063pose estimationimage matchinglocal featureglobal featuredeformable networkgeometric constraint |
| spellingShingle | Meng Xu Zhihuang Zhang Yuanhao Gong Stefan Poslad Regression-Based Camera Pose Estimation through Multi-Level Local Features and Global Features Sensors pose estimation image matching local feature global feature deformable network geometric constraint |
| title | Regression-Based Camera Pose Estimation through Multi-Level Local Features and Global Features |
| title_full | Regression-Based Camera Pose Estimation through Multi-Level Local Features and Global Features |
| title_fullStr | Regression-Based Camera Pose Estimation through Multi-Level Local Features and Global Features |
| title_full_unstemmed | Regression-Based Camera Pose Estimation through Multi-Level Local Features and Global Features |
| title_short | Regression-Based Camera Pose Estimation through Multi-Level Local Features and Global Features |
| title_sort | regression based camera pose estimation through multi level local features and global features |
| topic | pose estimation image matching local feature global feature deformable network geometric constraint |
| url | https://www.mdpi.com/1424-8220/23/8/4063 |
| work_keys_str_mv | AT mengxu regressionbasedcameraposeestimationthroughmultilevellocalfeaturesandglobalfeatures AT zhihuangzhang regressionbasedcameraposeestimationthroughmultilevellocalfeaturesandglobalfeatures AT yuanhaogong regressionbasedcameraposeestimationthroughmultilevellocalfeaturesandglobalfeatures AT stefanposlad regressionbasedcameraposeestimationthroughmultilevellocalfeaturesandglobalfeatures |