Instant uncertainty calibration of NERFs using a meta-calibrator
Neural Radiance Fields (NeRFs) have markedly improved novel view synthesis, but accurate uncertainty quantification in their image predictions remains an open problem. The prevailing methods for estimating uncertainty, including the state-of-the-art Density-aware NeRF Ensembles (DANE) [29], quantify...
| Main Authors: | , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
IEEE
2024
|
| _version_ | 1811140578471051264 |
|---|---|
| author | Amini-Naieni, N Jakab, T Vedaldi, A Clark, R |
| author_facet | Amini-Naieni, N Jakab, T Vedaldi, A Clark, R |
| author_sort | Amini-Naieni, N |
| collection | OXFORD |
| description | Neural Radiance Fields (NeRFs) have markedly improved
novel view synthesis, but accurate uncertainty quantification in their
image predictions remains an open problem. The prevailing methods
for estimating uncertainty, including the state-of-the-art Density-aware
NeRF Ensembles (DANE) [29], quantify uncertainty without calibration.
This frequently leads to over- or under-confidence in image predictions,
which can undermine their real-world applications. In this paper, we propose a method which, for the first time, achieves calibrated uncertainties
for NeRFs. To accomplish this, we overcome a significant challenge in
adapting existing calibration techniques to NeRFs: a need to hold out
ground truth images from the target scene, reducing the number of images left to train the NeRF. This issue is particularly problematic in
sparse-view settings, where we can operate with as few as three images. To address this, we introduce the concept of a meta-calibrator that
performs uncertainty calibration for NeRFs with a single forward pass
without the need for holding out any images from the target scene. Our
meta-calibrator is a neural network that takes as input the NeRF images
and uncalibrated uncertainty maps and outputs a scene-specific calibration curve that corrects the NeRF’s uncalibrated uncertainties. We show
that the meta-calibrator can generalize on unseen scenes and achieves
well-calibrated and state-of-the-art uncertainty for NeRFs, significantly
beating DANE and other approaches. This opens opportunities to improve applications that rely on accurate NeRF uncertainty estimates
such as next-best view planning and potentially more trustworthy image
reconstruction for medical diagnosis. |
| first_indexed | 2024-09-25T04:24:13Z |
| format | Conference item |
| id | oxford-uuid:d82c0b84-e689-4518-a1fb-b6cfe421a46a |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-09-25T04:24:13Z |
| publishDate | 2024 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:d82c0b84-e689-4518-a1fb-b6cfe421a46a2024-08-19T14:01:45ZInstant uncertainty calibration of NERFs using a meta-calibratorConference itemhttp://purl.org/coar/resource_type/c_5794uuid:d82c0b84-e689-4518-a1fb-b6cfe421a46aEnglishSymplectic ElementsIEEE2024Amini-Naieni, NJakab, TVedaldi, AClark, RNeural Radiance Fields (NeRFs) have markedly improved novel view synthesis, but accurate uncertainty quantification in their image predictions remains an open problem. The prevailing methods for estimating uncertainty, including the state-of-the-art Density-aware NeRF Ensembles (DANE) [29], quantify uncertainty without calibration. This frequently leads to over- or under-confidence in image predictions, which can undermine their real-world applications. In this paper, we propose a method which, for the first time, achieves calibrated uncertainties for NeRFs. To accomplish this, we overcome a significant challenge in adapting existing calibration techniques to NeRFs: a need to hold out ground truth images from the target scene, reducing the number of images left to train the NeRF. This issue is particularly problematic in sparse-view settings, where we can operate with as few as three images. To address this, we introduce the concept of a meta-calibrator that performs uncertainty calibration for NeRFs with a single forward pass without the need for holding out any images from the target scene. Our meta-calibrator is a neural network that takes as input the NeRF images and uncalibrated uncertainty maps and outputs a scene-specific calibration curve that corrects the NeRF’s uncalibrated uncertainties. We show that the meta-calibrator can generalize on unseen scenes and achieves well-calibrated and state-of-the-art uncertainty for NeRFs, significantly beating DANE and other approaches. This opens opportunities to improve applications that rely on accurate NeRF uncertainty estimates such as next-best view planning and potentially more trustworthy image reconstruction for medical diagnosis. |
| spellingShingle | Amini-Naieni, N Jakab, T Vedaldi, A Clark, R Instant uncertainty calibration of NERFs using a meta-calibrator |
| title | Instant uncertainty calibration of NERFs using a meta-calibrator |
| title_full | Instant uncertainty calibration of NERFs using a meta-calibrator |
| title_fullStr | Instant uncertainty calibration of NERFs using a meta-calibrator |
| title_full_unstemmed | Instant uncertainty calibration of NERFs using a meta-calibrator |
| title_short | Instant uncertainty calibration of NERFs using a meta-calibrator |
| title_sort | instant uncertainty calibration of nerfs using a meta calibrator |
| work_keys_str_mv | AT amininaienin instantuncertaintycalibrationofnerfsusingametacalibrator AT jakabt instantuncertaintycalibrationofnerfsusingametacalibrator AT vedaldia instantuncertaintycalibrationofnerfsusingametacalibrator AT clarkr instantuncertaintycalibrationofnerfsusingametacalibrator |