BENCHMARKING THE EXTRACTION OF 3D GEOMETRY FROM UAV IMAGES WITH DEEP LEARNING METHODS
3D reconstruction from single and multi-view stereo images is still an open research topic, despite the high number of solutions proposed in the last decades. The surge of deep learning methods has then stimulated the development of new methods using monocular (MDE, Monocular Depth Estimation), ster...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2023-10-01
|
| Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://isprs-archives.copernicus.org/articles/XLVIII-1-W3-2023/123/2023/isprs-archives-XLVIII-1-W3-2023-123-2023.pdf |
| _version_ | 1827789376812220416 |
|---|---|
| author | F. Nex N. Zhang F. Remondino E. M. Farella R. Qin C. Zhang |
| author_facet | F. Nex N. Zhang F. Remondino E. M. Farella R. Qin C. Zhang |
| author_sort | F. Nex |
| collection | DOAJ |
| description | 3D reconstruction from single and multi-view stereo images is still an open research topic, despite the high number of solutions proposed in the last decades. The surge of deep learning methods has then stimulated the development of new methods using monocular (MDE, Monocular Depth Estimation), stereoscopic and Multi-View Stereo (MVS) 3D reconstruction, showing promising results, often comparable to or even better than traditional methods. The more recent development of NeRF (Neural Radial Fields) has further triggered the interest for this kind of solution. Most of the proposed approaches, however, focus on terrestrial applications (e.g., autonomous driving or small artefacts 3D reconstructions), while airborne and UAV acquisitions are often overlooked. The recent introduction of new datasets, such as UseGeo has, therefore, given the opportunity to assess how state-of-the-art MDE, MVS and NeRF 3D reconstruction algorithms perform using airborne UAV images, allowing their comparison with LiDAR ground truth. This paper aims to present the results achieved by two MDE, two MVS and two NeRF approaches levering deep learning approaches, trained and tested using the UseGeo dataset. This work allows the comparison with a ground truth showing the current state of the art of these solutions and providing useful indications for their future development and improvement. |
| first_indexed | 2024-03-11T17:19:11Z |
| format | Article |
| id | doaj.art-fc2c917d9f2c4432bec06441225e3662 |
| institution | Directory Open Access Journal |
| issn | 1682-1750 2194-9034 |
| language | English |
| last_indexed | 2024-03-11T17:19:11Z |
| publishDate | 2023-10-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj.art-fc2c917d9f2c4432bec06441225e36622023-10-19T18:09:06ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342023-10-01XLVIII-1-W3-202312313010.5194/isprs-archives-XLVIII-1-W3-2023-123-2023BENCHMARKING THE EXTRACTION OF 3D GEOMETRY FROM UAV IMAGES WITH DEEP LEARNING METHODSF. Nex0N. Zhang1F. Remondino2E. M. Farella3R. Qin4C. Zhang5ITC Dep. of Earth Observation Science, University of Twente, Enschede, NetherlandsITC Dep. of Earth Observation Science, University of Twente, Enschede, Netherlands3D Optical Metrology (3DOM) unit, Bruno Kessler Foundation (FBK), Trento, Italy3D Optical Metrology (3DOM) unit, Bruno Kessler Foundation (FBK), Trento, ItalyGeospatial Data Analytics Laboratory, The Ohio State University, Columbus, USAGeospatial Data Analytics Laboratory, The Ohio State University, Columbus, USA3D reconstruction from single and multi-view stereo images is still an open research topic, despite the high number of solutions proposed in the last decades. The surge of deep learning methods has then stimulated the development of new methods using monocular (MDE, Monocular Depth Estimation), stereoscopic and Multi-View Stereo (MVS) 3D reconstruction, showing promising results, often comparable to or even better than traditional methods. The more recent development of NeRF (Neural Radial Fields) has further triggered the interest for this kind of solution. Most of the proposed approaches, however, focus on terrestrial applications (e.g., autonomous driving or small artefacts 3D reconstructions), while airborne and UAV acquisitions are often overlooked. The recent introduction of new datasets, such as UseGeo has, therefore, given the opportunity to assess how state-of-the-art MDE, MVS and NeRF 3D reconstruction algorithms perform using airborne UAV images, allowing their comparison with LiDAR ground truth. This paper aims to present the results achieved by two MDE, two MVS and two NeRF approaches levering deep learning approaches, trained and tested using the UseGeo dataset. This work allows the comparison with a ground truth showing the current state of the art of these solutions and providing useful indications for their future development and improvement.https://isprs-archives.copernicus.org/articles/XLVIII-1-W3-2023/123/2023/isprs-archives-XLVIII-1-W3-2023-123-2023.pdf |
| spellingShingle | F. Nex N. Zhang F. Remondino E. M. Farella R. Qin C. Zhang BENCHMARKING THE EXTRACTION OF 3D GEOMETRY FROM UAV IMAGES WITH DEEP LEARNING METHODS The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | BENCHMARKING THE EXTRACTION OF 3D GEOMETRY FROM UAV IMAGES WITH DEEP LEARNING METHODS |
| title_full | BENCHMARKING THE EXTRACTION OF 3D GEOMETRY FROM UAV IMAGES WITH DEEP LEARNING METHODS |
| title_fullStr | BENCHMARKING THE EXTRACTION OF 3D GEOMETRY FROM UAV IMAGES WITH DEEP LEARNING METHODS |
| title_full_unstemmed | BENCHMARKING THE EXTRACTION OF 3D GEOMETRY FROM UAV IMAGES WITH DEEP LEARNING METHODS |
| title_short | BENCHMARKING THE EXTRACTION OF 3D GEOMETRY FROM UAV IMAGES WITH DEEP LEARNING METHODS |
| title_sort | benchmarking the extraction of 3d geometry from uav images with deep learning methods |
| url | https://isprs-archives.copernicus.org/articles/XLVIII-1-W3-2023/123/2023/isprs-archives-XLVIII-1-W3-2023-123-2023.pdf |
| work_keys_str_mv | AT fnex benchmarkingtheextractionof3dgeometryfromuavimageswithdeeplearningmethods AT nzhang benchmarkingtheextractionof3dgeometryfromuavimageswithdeeplearningmethods AT fremondino benchmarkingtheextractionof3dgeometryfromuavimageswithdeeplearningmethods AT emfarella benchmarkingtheextractionof3dgeometryfromuavimageswithdeeplearningmethods AT rqin benchmarkingtheextractionof3dgeometryfromuavimageswithdeeplearningmethods AT czhang benchmarkingtheextractionof3dgeometryfromuavimageswithdeeplearningmethods |