Neural refinement for absolute pose regression with feature synthesis
Absolute Pose Regression (APR) methods use deep neural networks to directly regress camera poses from RGB images. Despite their advantages in inference speed and simplicity, these methods still fall short of the accuracy achieved by geometry-based techniques. To address this issue, we propose a new...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
IEEE
2024
|
| _version_ | 1826315012257349632 |
|---|---|
| author | Bhalgat, Y Li, X Bian, J Li, K Wang, Z Prisacariu, V Chen, S |
| author_facet | Bhalgat, Y Li, X Bian, J Li, K Wang, Z Prisacariu, V Chen, S |
| author_sort | Bhalgat, Y |
| collection | OXFORD |
| description | Absolute Pose Regression (APR) methods use deep neural networks to directly regress camera poses from RGB
images. Despite their advantages in inference speed and
simplicity, these methods still fall short of the accuracy
achieved by geometry-based techniques. To address this issue, we propose a new model called the Neural Feature Synthesizer (NeFeS). Our approach encodes 3D geometric features during training and renders dense novel view features
at test time to refine estimated camera poses from arbitrary
APR methods. Unlike previous APR works that require additional unlabeled training data, our method leverages implicit geometric constraints during test time using a robust
feature field. To enhance the robustness of our NeFeS network, we introduce a feature fusion module and a progressive training strategy. Our proposed method improves the
state-of-the-art single-image APR accuracy by as much as
54.9% on indoor and outdoor benchmark datasets without
additional time-consuming unlabeled data training. |
| first_indexed | 2024-03-07T07:39:13Z |
| format | Conference item |
| id | oxford-uuid:526e1254-f781-4285-9d97-8e0659e0328f |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-12-09T03:16:09Z |
| publishDate | 2024 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:526e1254-f781-4285-9d97-8e0659e0328f2024-10-25T09:02:17ZNeural refinement for absolute pose regression with feature synthesisConference itemhttp://purl.org/coar/resource_type/c_5794uuid:526e1254-f781-4285-9d97-8e0659e0328fEnglishSymplectic ElementsIEEE2024Bhalgat, YLi, XBian, JLi, KWang, ZPrisacariu, VChen, SAbsolute Pose Regression (APR) methods use deep neural networks to directly regress camera poses from RGB images. Despite their advantages in inference speed and simplicity, these methods still fall short of the accuracy achieved by geometry-based techniques. To address this issue, we propose a new model called the Neural Feature Synthesizer (NeFeS). Our approach encodes 3D geometric features during training and renders dense novel view features at test time to refine estimated camera poses from arbitrary APR methods. Unlike previous APR works that require additional unlabeled training data, our method leverages implicit geometric constraints during test time using a robust feature field. To enhance the robustness of our NeFeS network, we introduce a feature fusion module and a progressive training strategy. Our proposed method improves the state-of-the-art single-image APR accuracy by as much as 54.9% on indoor and outdoor benchmark datasets without additional time-consuming unlabeled data training. |
| spellingShingle | Bhalgat, Y Li, X Bian, J Li, K Wang, Z Prisacariu, V Chen, S Neural refinement for absolute pose regression with feature synthesis |
| title | Neural refinement for absolute pose regression with feature synthesis |
| title_full | Neural refinement for absolute pose regression with feature synthesis |
| title_fullStr | Neural refinement for absolute pose regression with feature synthesis |
| title_full_unstemmed | Neural refinement for absolute pose regression with feature synthesis |
| title_short | Neural refinement for absolute pose regression with feature synthesis |
| title_sort | neural refinement for absolute pose regression with feature synthesis |
| work_keys_str_mv | AT bhalgaty neuralrefinementforabsoluteposeregressionwithfeaturesynthesis AT lix neuralrefinementforabsoluteposeregressionwithfeaturesynthesis AT bianj neuralrefinementforabsoluteposeregressionwithfeaturesynthesis AT lik neuralrefinementforabsoluteposeregressionwithfeaturesynthesis AT wangz neuralrefinementforabsoluteposeregressionwithfeaturesynthesis AT prisacariuv neuralrefinementforabsoluteposeregressionwithfeaturesynthesis AT chens neuralrefinementforabsoluteposeregressionwithfeaturesynthesis |