DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo Networks
We propose an enhancement module called depth discontinuity learning (DDL) for learning-based multi-view stereo (MVS) methods. Traditional methods are known for their accuracy but struggle with completeness. While recent learning-based methods have improved completeness at the cost of accuracy, our...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/12/2970 |
| _version_ | 1797592845050183680 |
|---|---|
| author | Nail Ibrahimli Hugo Ledoux Julian F. P. Kooij Liangliang Nan |
| author_facet | Nail Ibrahimli Hugo Ledoux Julian F. P. Kooij Liangliang Nan |
| author_sort | Nail Ibrahimli |
| collection | DOAJ |
| description | We propose an enhancement module called depth discontinuity learning (DDL) for learning-based multi-view stereo (MVS) methods. Traditional methods are known for their accuracy but struggle with completeness. While recent learning-based methods have improved completeness at the cost of accuracy, our DDL approach aims to improve accuracy while retaining completeness in the reconstruction process. To achieve this, we introduce the joint estimation of depth and boundary maps, where the boundary maps are explicitly utilized for further refinement of the depth maps. We validate our idea by integrating it into an existing learning-based MVS pipeline where the reconstruction depends on high-quality depth map estimation. Extensive experiments on various datasets, namely DTU, ETH3D, “Tanks and Temples”, and BlendedMVS, show that our method improves reconstruction quality compared to our baseline, Patchmatchnet. Our ablation study demonstrates that incorporating the proposed DDL significantly reduces the depth map error, for instance, by more than 30% on the DTU dataset, and leads to improved depth map quality in both smooth and boundary regions. Additionally, our qualitative analysis has shown that the reconstructed point cloud exhibits enhanced quality without any significant compromise on completeness. Finally, the experiments reveal that our proposed model and strategies exhibit strong generalization capabilities across the various datasets. |
| first_indexed | 2024-03-11T01:58:53Z |
| format | Article |
| id | doaj.art-6363a1e1ccef4f8289b528048b35e467 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-11T01:58:53Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-6363a1e1ccef4f8289b528048b35e4672023-11-18T12:24:29ZengMDPI AGRemote Sensing2072-42922023-06-011512297010.3390/rs15122970DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo NetworksNail Ibrahimli0Hugo Ledoux1Julian F. P. Kooij2Liangliang Nan33D Geoinformation, Faculty of Architecture and the Built Environment, Delft University of Technology, 2628 BL Delft, The Netherlands3D Geoinformation, Faculty of Architecture and the Built Environment, Delft University of Technology, 2628 BL Delft, The NetherlandsIntelligent Vehicles Group, Delft University of Technology, 2628 BL Delft, The Netherlands3D Geoinformation, Faculty of Architecture and the Built Environment, Delft University of Technology, 2628 BL Delft, The NetherlandsWe propose an enhancement module called depth discontinuity learning (DDL) for learning-based multi-view stereo (MVS) methods. Traditional methods are known for their accuracy but struggle with completeness. While recent learning-based methods have improved completeness at the cost of accuracy, our DDL approach aims to improve accuracy while retaining completeness in the reconstruction process. To achieve this, we introduce the joint estimation of depth and boundary maps, where the boundary maps are explicitly utilized for further refinement of the depth maps. We validate our idea by integrating it into an existing learning-based MVS pipeline where the reconstruction depends on high-quality depth map estimation. Extensive experiments on various datasets, namely DTU, ETH3D, “Tanks and Temples”, and BlendedMVS, show that our method improves reconstruction quality compared to our baseline, Patchmatchnet. Our ablation study demonstrates that incorporating the proposed DDL significantly reduces the depth map error, for instance, by more than 30% on the DTU dataset, and leads to improved depth map quality in both smooth and boundary regions. Additionally, our qualitative analysis has shown that the reconstructed point cloud exhibits enhanced quality without any significant compromise on completeness. Finally, the experiments reveal that our proposed model and strategies exhibit strong generalization capabilities across the various datasets.https://www.mdpi.com/2072-4292/15/12/2970multi-view stereo3D reconstructiondepth map refinementdepth boundary estimation |
| spellingShingle | Nail Ibrahimli Hugo Ledoux Julian F. P. Kooij Liangliang Nan DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo Networks Remote Sensing multi-view stereo 3D reconstruction depth map refinement depth boundary estimation |
| title | DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo Networks |
| title_full | DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo Networks |
| title_fullStr | DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo Networks |
| title_full_unstemmed | DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo Networks |
| title_short | DDL-MVS: Depth Discontinuity Learning for Multi-View Stereo Networks |
| title_sort | ddl mvs depth discontinuity learning for multi view stereo networks |
| topic | multi-view stereo 3D reconstruction depth map refinement depth boundary estimation |
| url | https://www.mdpi.com/2072-4292/15/12/2970 |
| work_keys_str_mv | AT nailibrahimli ddlmvsdepthdiscontinuitylearningformultiviewstereonetworks AT hugoledoux ddlmvsdepthdiscontinuitylearningformultiviewstereonetworks AT julianfpkooij ddlmvsdepthdiscontinuitylearningformultiviewstereonetworks AT liangliangnan ddlmvsdepthdiscontinuitylearningformultiviewstereonetworks |