Refinement for absolute pose regression with neural feature synthesis
Absolute Pose Regression (APR) methods use deep neural networks to directly regress camera poses from RGB images. Despite their advantages in inference speed and simplicity, these methods still fall short of the accuracy achieved by geometry-based techniques. To address this issue, we propose a new...
| Hlavní autoři: | , , , , , , |
|---|---|
| Médium: | Internet publication |
| Jazyk: | English |
| Vydáno: |
arXiv
2023
|
| _version_ | 1826311804754591744 |
|---|---|
| author | Chen, S Bhalgat, Y Li, X Bian, J Li, K Wang, Z Prisacariu, V |
| author_facet | Chen, S Bhalgat, Y Li, X Bian, J Li, K Wang, Z Prisacariu, V |
| author_sort | Chen, S |
| collection | OXFORD |
| description | Absolute Pose Regression (APR) methods use deep neural networks to directly regress camera poses from RGB images. Despite their advantages in inference speed and simplicity, these methods still fall short of the accuracy achieved by geometry-based techniques. To address this issue, we propose a new model called the Neural Feature Synthesizer (NeFeS). Our approach encodes 3D geometric features during training and renders dense novel view features at test time to refine estimated camera poses from arbitrary APR methods. Unlike previous APR works that require additional unlabeled training data, our method leverages implicit geometric constraints during test time using a robust feature field. To enhance the robustness of our NeFeS network, we introduce a feature fusion module and a progressive training strategy. Our proposed method improves the state-of-the-art single-image APR accuracy by as much as 54.9% on indoor and outdoor benchmark datasets without additional time-consuming unlabeled data training. |
| first_indexed | 2024-03-07T08:16:39Z |
| format | Internet publication |
| id | oxford-uuid:19bccf1c-7d02-4aa5-884e-cec9d4a963c6 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T08:16:39Z |
| publishDate | 2023 |
| publisher | arXiv |
| record_format | dspace |
| spelling | oxford-uuid:19bccf1c-7d02-4aa5-884e-cec9d4a963c62024-01-11T14:00:01ZRefinement for absolute pose regression with neural feature synthesisInternet publicationhttp://purl.org/coar/resource_type/c_7ad9uuid:19bccf1c-7d02-4aa5-884e-cec9d4a963c6EnglishSymplectic ElementsarXiv2023Chen, SBhalgat, YLi, XBian, JLi, KWang, ZPrisacariu, VAbsolute Pose Regression (APR) methods use deep neural networks to directly regress camera poses from RGB images. Despite their advantages in inference speed and simplicity, these methods still fall short of the accuracy achieved by geometry-based techniques. To address this issue, we propose a new model called the Neural Feature Synthesizer (NeFeS). Our approach encodes 3D geometric features during training and renders dense novel view features at test time to refine estimated camera poses from arbitrary APR methods. Unlike previous APR works that require additional unlabeled training data, our method leverages implicit geometric constraints during test time using a robust feature field. To enhance the robustness of our NeFeS network, we introduce a feature fusion module and a progressive training strategy. Our proposed method improves the state-of-the-art single-image APR accuracy by as much as 54.9% on indoor and outdoor benchmark datasets without additional time-consuming unlabeled data training. |
| spellingShingle | Chen, S Bhalgat, Y Li, X Bian, J Li, K Wang, Z Prisacariu, V Refinement for absolute pose regression with neural feature synthesis |
| title | Refinement for absolute pose regression with neural feature synthesis |
| title_full | Refinement for absolute pose regression with neural feature synthesis |
| title_fullStr | Refinement for absolute pose regression with neural feature synthesis |
| title_full_unstemmed | Refinement for absolute pose regression with neural feature synthesis |
| title_short | Refinement for absolute pose regression with neural feature synthesis |
| title_sort | refinement for absolute pose regression with neural feature synthesis |
| work_keys_str_mv | AT chens refinementforabsoluteposeregressionwithneuralfeaturesynthesis AT bhalgaty refinementforabsoluteposeregressionwithneuralfeaturesynthesis AT lix refinementforabsoluteposeregressionwithneuralfeaturesynthesis AT bianj refinementforabsoluteposeregressionwithneuralfeaturesynthesis AT lik refinementforabsoluteposeregressionwithneuralfeaturesynthesis AT wangz refinementforabsoluteposeregressionwithneuralfeaturesynthesis AT prisacariuv refinementforabsoluteposeregressionwithneuralfeaturesynthesis |