UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering Scheme
While recent advances in the field of neural rendering have shown impressive 3D reconstruction performance, it is still a challenge to accurately capture the appearance and geometry of a scene by using neural rendering, especially for remote sensing scenes. This is because both rendering methods, i....
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/18/4634 |
| _version_ | 1827723734380707840 |
|---|---|
| author | Yiming Yan Weikun Zhou Nan Su Chi Zhang |
| author_facet | Yiming Yan Weikun Zhou Nan Su Chi Zhang |
| author_sort | Yiming Yan |
| collection | DOAJ |
| description | While recent advances in the field of neural rendering have shown impressive 3D reconstruction performance, it is still a challenge to accurately capture the appearance and geometry of a scene by using neural rendering, especially for remote sensing scenes. This is because both rendering methods, i.e., surface rendering and volume rendering, have their own limitations. Furthermore, when neural rendering is applied to remote sensing scenes, the view sparsity and content complexity that characterize these scenes will severely hinder its performance. In this work, we aim to address these challenges and to make neural rendering techniques available for 3D reconstruction in remote sensing environments. To achieve this, we propose a novel 3D surface reconstruction method called UniRender. UniRender offers three improvements in locating an accurate 3D surface by using neural rendering: (1) unifying surface and volume rendering by employing their strengths while discarding their weaknesses, which enables accurate 3D surface position localization in a coarse-to-fine manner; (2) incorporating photometric consistency constraints during rendering, and utilizing the points reconstructed by structure from motion (SFM) or multi-view stereo (MVS), to constrain reconstructed surfaces, which significantly improves the accuracy of 3D reconstruction; (3) improving the sampling strategy by locating sampling points in the foreground regions where the surface needs to be reconstructed, thus obtaining better detail in the reconstruction results. Extensive experiments demonstrate that UniRender can reconstruct high-quality 3D surfaces in various remote sensing scenes. |
| first_indexed | 2024-03-10T22:03:54Z |
| format | Article |
| id | doaj.art-04bac41bcc1a415aaabdfa67dbe9d081 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T22:03:54Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-04bac41bcc1a415aaabdfa67dbe9d0812023-11-19T12:50:30ZengMDPI AGRemote Sensing2072-42922023-09-011518463410.3390/rs15184634UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering SchemeYiming Yan0Weikun Zhou1Nan Su2Chi Zhang3College of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, ChinaCollege of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, ChinaCollege of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, ChinaCollege of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, ChinaWhile recent advances in the field of neural rendering have shown impressive 3D reconstruction performance, it is still a challenge to accurately capture the appearance and geometry of a scene by using neural rendering, especially for remote sensing scenes. This is because both rendering methods, i.e., surface rendering and volume rendering, have their own limitations. Furthermore, when neural rendering is applied to remote sensing scenes, the view sparsity and content complexity that characterize these scenes will severely hinder its performance. In this work, we aim to address these challenges and to make neural rendering techniques available for 3D reconstruction in remote sensing environments. To achieve this, we propose a novel 3D surface reconstruction method called UniRender. UniRender offers three improvements in locating an accurate 3D surface by using neural rendering: (1) unifying surface and volume rendering by employing their strengths while discarding their weaknesses, which enables accurate 3D surface position localization in a coarse-to-fine manner; (2) incorporating photometric consistency constraints during rendering, and utilizing the points reconstructed by structure from motion (SFM) or multi-view stereo (MVS), to constrain reconstructed surfaces, which significantly improves the accuracy of 3D reconstruction; (3) improving the sampling strategy by locating sampling points in the foreground regions where the surface needs to be reconstructed, thus obtaining better detail in the reconstruction results. Extensive experiments demonstrate that UniRender can reconstruct high-quality 3D surfaces in various remote sensing scenes.https://www.mdpi.com/2072-4292/15/18/46343D reconstructionsurface reconstructionaerial imagesrenderingimplicit representationsigned distance field |
| spellingShingle | Yiming Yan Weikun Zhou Nan Su Chi Zhang UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering Scheme Remote Sensing 3D reconstruction surface reconstruction aerial images rendering implicit representation signed distance field |
| title | UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering Scheme |
| title_full | UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering Scheme |
| title_fullStr | UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering Scheme |
| title_full_unstemmed | UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering Scheme |
| title_short | UniRender: Reconstructing 3D Surfaces from Aerial Images with a Unified Rendering Scheme |
| title_sort | unirender reconstructing 3d surfaces from aerial images with a unified rendering scheme |
| topic | 3D reconstruction surface reconstruction aerial images rendering implicit representation signed distance field |
| url | https://www.mdpi.com/2072-4292/15/18/4634 |
| work_keys_str_mv | AT yimingyan unirenderreconstructing3dsurfacesfromaerialimageswithaunifiedrenderingscheme AT weikunzhou unirenderreconstructing3dsurfacesfromaerialimageswithaunifiedrenderingscheme AT nansu unirenderreconstructing3dsurfacesfromaerialimageswithaunifiedrenderingscheme AT chizhang unirenderreconstructing3dsurfacesfromaerialimageswithaunifiedrenderingscheme |