Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View Reconstruction
Surface reconstruction using neural networks has proven effective in reconstructing dense 3D surfaces through image-based neural rendering. Nevertheless, current methods are challenging when dealing with the intricate details of large-scale scenes. The high-fidelity reconstruction performance of neu...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/3/881 |
| _version_ | 1797318189759070208 |
|---|---|
| author | Wei Wang Fengjiao Gao Yongliang Shen |
| author_facet | Wei Wang Fengjiao Gao Yongliang Shen |
| author_sort | Wei Wang |
| collection | DOAJ |
| description | Surface reconstruction using neural networks has proven effective in reconstructing dense 3D surfaces through image-based neural rendering. Nevertheless, current methods are challenging when dealing with the intricate details of large-scale scenes. The high-fidelity reconstruction performance of neural rendering is constrained by the view sparsity and structural complexity of such scenes. In this paper, we present Res-NeuS, a method combining ResNet-50 and neural surface rendering for dense 3D reconstruction. Specifically, we present appearance embeddings: ResNet-50 is used to extract the appearance depth features of an image to further capture more scene details. We interpolate points near the surface and optimize their weights for the accurate localization of 3D surfaces. We introduce photometric consistency and geometric constraints to optimize 3D surfaces and eliminate geometric ambiguity existing in current methods. Finally, we design a 3D geometry automatic sampling to filter out uninteresting areas and reconstruct complex surface details in a coarse-to-fine manner. Comprehensive experiments demonstrate Res-NeuS’s superior capability in the reconstruction of 3D surfaces in complex, large-scale scenes, and the harmful distance of the reconstructed 3D model is 0.4 times that of general neural rendering 3D reconstruction methods and 0.6 times that of traditional 3D reconstruction methods. |
| first_indexed | 2024-03-08T03:48:51Z |
| format | Article |
| id | doaj.art-77b74a7385ea494fabbed0e60e0947b2 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-08T03:48:51Z |
| publishDate | 2024-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-77b74a7385ea494fabbed0e60e0947b22024-02-09T15:22:08ZengMDPI AGSensors1424-82202024-01-0124388110.3390/s24030881Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View ReconstructionWei Wang0Fengjiao Gao1Yongliang Shen2Intelligent Manufacturing Institute, Heilongjiang Academy of Sciences, Harbin 150090, ChinaIntelligent Manufacturing Institute, Heilongjiang Academy of Sciences, Harbin 150090, ChinaCollege of Electronic Engineering, Heilongjiang University, Harbin 150080, ChinaSurface reconstruction using neural networks has proven effective in reconstructing dense 3D surfaces through image-based neural rendering. Nevertheless, current methods are challenging when dealing with the intricate details of large-scale scenes. The high-fidelity reconstruction performance of neural rendering is constrained by the view sparsity and structural complexity of such scenes. In this paper, we present Res-NeuS, a method combining ResNet-50 and neural surface rendering for dense 3D reconstruction. Specifically, we present appearance embeddings: ResNet-50 is used to extract the appearance depth features of an image to further capture more scene details. We interpolate points near the surface and optimize their weights for the accurate localization of 3D surfaces. We introduce photometric consistency and geometric constraints to optimize 3D surfaces and eliminate geometric ambiguity existing in current methods. Finally, we design a 3D geometry automatic sampling to filter out uninteresting areas and reconstruct complex surface details in a coarse-to-fine manner. Comprehensive experiments demonstrate Res-NeuS’s superior capability in the reconstruction of 3D surfaces in complex, large-scale scenes, and the harmful distance of the reconstructed 3D model is 0.4 times that of general neural rendering 3D reconstruction methods and 0.6 times that of traditional 3D reconstruction methods.https://www.mdpi.com/1424-8220/24/3/881surface reconstructionneural radiance fieldrenderingResNet-50appearance embedding |
| spellingShingle | Wei Wang Fengjiao Gao Yongliang Shen Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View Reconstruction Sensors surface reconstruction neural radiance field rendering ResNet-50 appearance embedding |
| title | Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View Reconstruction |
| title_full | Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View Reconstruction |
| title_fullStr | Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View Reconstruction |
| title_full_unstemmed | Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View Reconstruction |
| title_short | Res-NeuS: Deep Residuals and Neural Implicit Surface Learning for Multi-View Reconstruction |
| title_sort | res neus deep residuals and neural implicit surface learning for multi view reconstruction |
| topic | surface reconstruction neural radiance field rendering ResNet-50 appearance embedding |
| url | https://www.mdpi.com/1424-8220/24/3/881 |
| work_keys_str_mv | AT weiwang resneusdeepresidualsandneuralimplicitsurfacelearningformultiviewreconstruction AT fengjiaogao resneusdeepresidualsandneuralimplicitsurfacelearningformultiviewreconstruction AT yongliangshen resneusdeepresidualsandneuralimplicitsurfacelearningformultiviewreconstruction |