NeRF--: Neural Radiance Fields without known camera parameters
Considering the problem of novel view synthesis (NVS) from only a set of 2D images, we simplify the training process of Neural Radiance Field (NeRF) on forward-facing scenes by removing the requirement of known or pre-computed camera parameters, including both intrinsics and 6DoF poses. To this end,...
| Main Authors: | , , , , |
|---|---|
| Format: | Internet publication |
| Language: | English |
| Published: |
2021
|
| _version_ | 1811139634577539072 |
|---|---|
| author | Wang, Z Wu, S Xie, W Chen, M Prisacariu, VA |
| author_facet | Wang, Z Wu, S Xie, W Chen, M Prisacariu, VA |
| author_sort | Wang, Z |
| collection | OXFORD |
| description | Considering the problem of novel view synthesis (NVS) from only a set of 2D images, we simplify the training process of Neural Radiance Field (NeRF) on forward-facing scenes by removing the requirement of known or pre-computed camera parameters, including both intrinsics and 6DoF poses. To this end, we propose NeRF−−, with three contributions: First, we show that the camera parameters can be jointly optimised as learnable parameters with NeRF training, through a photometric reconstruction; Second, to benchmark the camera parameter estimation and the quality of novel view renderings, we introduce a new dataset of path-traced synthetic scenes, termed as Blender Forward-Facing Dataset (BLEFF); Third, we conduct extensive analyses to understand the training behaviours under various camera motions, and show that in most scenarios, the joint optimisation pipeline can recover accurate camera parameters and achieve comparable novel view synthesis quality as those trained with COLMAP pre-computed camera parameters. Our code and data are available at https://nerfmm.active.vision. |
| first_indexed | 2024-09-25T04:09:12Z |
| format | Internet publication |
| id | oxford-uuid:9d743744-16bc-4828-9510-fc69257c9554 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-09-25T04:09:12Z |
| publishDate | 2021 |
| record_format | dspace |
| spelling | oxford-uuid:9d743744-16bc-4828-9510-fc69257c95542024-06-11T16:02:49ZNeRF--: Neural Radiance Fields without known camera parametersInternet publicationhttp://purl.org/coar/resource_type/c_7ad9uuid:9d743744-16bc-4828-9510-fc69257c9554EnglishSymplectic Elements2021Wang, ZWu, SXie, WChen, MPrisacariu, VAConsidering the problem of novel view synthesis (NVS) from only a set of 2D images, we simplify the training process of Neural Radiance Field (NeRF) on forward-facing scenes by removing the requirement of known or pre-computed camera parameters, including both intrinsics and 6DoF poses. To this end, we propose NeRF−−, with three contributions: First, we show that the camera parameters can be jointly optimised as learnable parameters with NeRF training, through a photometric reconstruction; Second, to benchmark the camera parameter estimation and the quality of novel view renderings, we introduce a new dataset of path-traced synthetic scenes, termed as Blender Forward-Facing Dataset (BLEFF); Third, we conduct extensive analyses to understand the training behaviours under various camera motions, and show that in most scenarios, the joint optimisation pipeline can recover accurate camera parameters and achieve comparable novel view synthesis quality as those trained with COLMAP pre-computed camera parameters. Our code and data are available at https://nerfmm.active.vision. |
| spellingShingle | Wang, Z Wu, S Xie, W Chen, M Prisacariu, VA NeRF--: Neural Radiance Fields without known camera parameters |
| title | NeRF--: Neural Radiance Fields without known camera parameters |
| title_full | NeRF--: Neural Radiance Fields without known camera parameters |
| title_fullStr | NeRF--: Neural Radiance Fields without known camera parameters |
| title_full_unstemmed | NeRF--: Neural Radiance Fields without known camera parameters |
| title_short | NeRF--: Neural Radiance Fields without known camera parameters |
| title_sort | nerf neural radiance fields without known camera parameters |
| work_keys_str_mv | AT wangz nerfneuralradiancefieldswithoutknowncameraparameters AT wus nerfneuralradiancefieldswithoutknowncameraparameters AT xiew nerfneuralradiancefieldswithoutknowncameraparameters AT chenm nerfneuralradiancefieldswithoutknowncameraparameters AT prisacariuva nerfneuralradiancefieldswithoutknowncameraparameters |