SEMANTIC URBAN MESH SEGMENTATION BASED ON AERIAL OBLIQUE IMAGES AND POINT CLOUDS USING DEEP LEARNING
The use of deep machine learning methods for semantic classification of city mesh models is one of the current trends in geoscience development. Thanks to the thriving development of Convolutional Neural Networks (CNNs) it is now achievable to conduct fully automated process of building aforemention...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2022-05-01
|
| Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2022/485/2022/isprs-archives-XLIII-B2-2022-485-2022.pdf |
| _version_ | 1811232022221291520 |
|---|---|
| author | Ł. Wilk Ł. Wilk D. Mielczarek W. Ostrowski W. Dominik J. Krawczyk |
| author_facet | Ł. Wilk Ł. Wilk D. Mielczarek W. Ostrowski W. Dominik J. Krawczyk |
| author_sort | Ł. Wilk |
| collection | DOAJ |
| description | The use of deep machine learning methods for semantic classification of city mesh models is one of the current trends in geoscience development. Thanks to the thriving development of Convolutional Neural Networks (CNNs) it is now achievable to conduct fully automated process of building aforementioned 3D model by means of photogrammetric techniques and supplement it with additional semantic information obtained by Artificial Intelligence (AI) algorithms. In order to guarantee the comprehensiveness of said information it is essential to use an extensive range of 3D data including oblique aerial imagery and aerial laser scanning (ALS). Such comprehensive 3D mesh models may be later implemented in many Digital Twin class solutions additionally supported with modern GIS systems and its algorithms. To proof the validity of this thesis, the article showcases results of research conducted using deep learning based solutions tested on two datasets - real-world data in the form of oblique aerial images and ALS point clouds acquired in Bordeaux, France using novel Leica CityMapper-1 multisensoral system and large-scale dataset from SUM: A Benchmark Dataset of Semantic Urban Meshes. Both subalgorithms make use of CNNs as its core-feature. To perform accurate classification of oblique aerial scenes PSP-Net architecture accelerated by techniques of transfer learning has been used. Second algorithm destined for ALS point clouds utilizes CNN as well, but in this case implementation is based on proprietary architecture. The results of the experiments demonstrate that the utilizing these two mutually complementary solutions to extract new semantic information for city mesh models in proposed manner compared with the state-of-the-art methods achieves competitive classification performance. |
| first_indexed | 2024-04-12T10:55:01Z |
| format | Article |
| id | doaj.art-005163df081b411193bc938f353af436 |
| institution | Directory Open Access Journal |
| issn | 1682-1750 2194-9034 |
| language | English |
| last_indexed | 2024-04-12T10:55:01Z |
| publishDate | 2022-05-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj.art-005163df081b411193bc938f353af4362022-12-22T03:36:06ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342022-05-01XLIII-B2-202248549110.5194/isprs-archives-XLIII-B2-2022-485-2022SEMANTIC URBAN MESH SEGMENTATION BASED ON AERIAL OBLIQUE IMAGES AND POINT CLOUDS USING DEEP LEARNINGŁ. Wilk0Ł. Wilk1D. Mielczarek2W. Ostrowski3W. Dominik4J. Krawczyk5Department of Photogrammetry, Remote Sensing and Spatial Information Systems, Faculty of Geodesy and Cartography, Warsaw University of Technology, Warsaw, PolandOPEGIEKA, Elbląg, PolandOPEGIEKA, Elbląg, PolandDepartment of Photogrammetry, Remote Sensing and Spatial Information Systems, Faculty of Geodesy and Cartography, Warsaw University of Technology, Warsaw, PolandOPEGIEKA, Elbląg, PolandOPEGIEKA, Elbląg, PolandThe use of deep machine learning methods for semantic classification of city mesh models is one of the current trends in geoscience development. Thanks to the thriving development of Convolutional Neural Networks (CNNs) it is now achievable to conduct fully automated process of building aforementioned 3D model by means of photogrammetric techniques and supplement it with additional semantic information obtained by Artificial Intelligence (AI) algorithms. In order to guarantee the comprehensiveness of said information it is essential to use an extensive range of 3D data including oblique aerial imagery and aerial laser scanning (ALS). Such comprehensive 3D mesh models may be later implemented in many Digital Twin class solutions additionally supported with modern GIS systems and its algorithms. To proof the validity of this thesis, the article showcases results of research conducted using deep learning based solutions tested on two datasets - real-world data in the form of oblique aerial images and ALS point clouds acquired in Bordeaux, France using novel Leica CityMapper-1 multisensoral system and large-scale dataset from SUM: A Benchmark Dataset of Semantic Urban Meshes. Both subalgorithms make use of CNNs as its core-feature. To perform accurate classification of oblique aerial scenes PSP-Net architecture accelerated by techniques of transfer learning has been used. Second algorithm destined for ALS point clouds utilizes CNN as well, but in this case implementation is based on proprietary architecture. The results of the experiments demonstrate that the utilizing these two mutually complementary solutions to extract new semantic information for city mesh models in proposed manner compared with the state-of-the-art methods achieves competitive classification performance.https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2022/485/2022/isprs-archives-XLIII-B2-2022-485-2022.pdf |
| spellingShingle | Ł. Wilk Ł. Wilk D. Mielczarek W. Ostrowski W. Dominik J. Krawczyk SEMANTIC URBAN MESH SEGMENTATION BASED ON AERIAL OBLIQUE IMAGES AND POINT CLOUDS USING DEEP LEARNING The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | SEMANTIC URBAN MESH SEGMENTATION BASED ON AERIAL OBLIQUE IMAGES AND POINT CLOUDS USING DEEP LEARNING |
| title_full | SEMANTIC URBAN MESH SEGMENTATION BASED ON AERIAL OBLIQUE IMAGES AND POINT CLOUDS USING DEEP LEARNING |
| title_fullStr | SEMANTIC URBAN MESH SEGMENTATION BASED ON AERIAL OBLIQUE IMAGES AND POINT CLOUDS USING DEEP LEARNING |
| title_full_unstemmed | SEMANTIC URBAN MESH SEGMENTATION BASED ON AERIAL OBLIQUE IMAGES AND POINT CLOUDS USING DEEP LEARNING |
| title_short | SEMANTIC URBAN MESH SEGMENTATION BASED ON AERIAL OBLIQUE IMAGES AND POINT CLOUDS USING DEEP LEARNING |
| title_sort | semantic urban mesh segmentation based on aerial oblique images and point clouds using deep learning |
| url | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2022/485/2022/isprs-archives-XLIII-B2-2022-485-2022.pdf |
| work_keys_str_mv | AT łwilk semanticurbanmeshsegmentationbasedonaerialobliqueimagesandpointcloudsusingdeeplearning AT łwilk semanticurbanmeshsegmentationbasedonaerialobliqueimagesandpointcloudsusingdeeplearning AT dmielczarek semanticurbanmeshsegmentationbasedonaerialobliqueimagesandpointcloudsusingdeeplearning AT wostrowski semanticurbanmeshsegmentationbasedonaerialobliqueimagesandpointcloudsusingdeeplearning AT wdominik semanticurbanmeshsegmentationbasedonaerialobliqueimagesandpointcloudsusingdeeplearning AT jkrawczyk semanticurbanmeshsegmentationbasedonaerialobliqueimagesandpointcloudsusingdeeplearning |