Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry
Developments in the field of artificial intelligence have made great strides in the field of automatic semantic segmentation, both in the 2D (image) and 3D spaces. Within the context of 3D recording technology it has also seen application in several areas, most notably in creating semantically rich...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/3/966 |
| _version_ | 1797484734510530560 |
|---|---|
| author | Arnadi Murtiyoso Eugenio Pellis Pierre Grussenmeyer Tania Landes Andrea Masiero |
| author_facet | Arnadi Murtiyoso Eugenio Pellis Pierre Grussenmeyer Tania Landes Andrea Masiero |
| author_sort | Arnadi Murtiyoso |
| collection | DOAJ |
| description | Developments in the field of artificial intelligence have made great strides in the field of automatic semantic segmentation, both in the 2D (image) and 3D spaces. Within the context of 3D recording technology it has also seen application in several areas, most notably in creating semantically rich point clouds which is usually performed manually. In this paper, we propose the introduction of deep learning-based semantic image segmentation into the photogrammetric 3D reconstruction and classification workflow. The main objective is to be able to introduce semantic classification at the beginning of the classical photogrammetric workflow in order to automatically create classified dense point clouds by the end of the said workflow. In this regard, automatic image masking depending on pre-determined classes were performed using a previously trained neural network. The image masks were then employed during dense image matching in order to constraint the process into the respective classes, thus automatically creating semantically classified point clouds as the final output. Results show that the developed method is promising, with automation of the whole process feasible from input (images) to output (labelled point clouds). Quantitative assessment gave good results for specific classes e.g., building facades and windows, with IoU scores of 0.79 and 0.77 respectively. |
| first_indexed | 2024-03-09T23:08:42Z |
| format | Article |
| id | doaj.art-3e8235a331364ce3a75df7d4b10512f4 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T23:08:42Z |
| publishDate | 2022-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-3e8235a331364ce3a75df7d4b10512f42023-11-23T17:48:20ZengMDPI AGSensors1424-82202022-01-0122396610.3390/s22030966Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range PhotogrammetryArnadi Murtiyoso0Eugenio Pellis1Pierre Grussenmeyer2Tania Landes3Andrea Masiero4Université de Strasbourg, INSA Strasbourg, CNRS, ICube Laboratory UMR 7357, 67084 Strasbourg, FranceUniversité de Strasbourg, INSA Strasbourg, CNRS, ICube Laboratory UMR 7357, 67084 Strasbourg, FranceUniversité de Strasbourg, INSA Strasbourg, CNRS, ICube Laboratory UMR 7357, 67084 Strasbourg, FranceUniversité de Strasbourg, INSA Strasbourg, CNRS, ICube Laboratory UMR 7357, 67084 Strasbourg, FranceDepartment of Civil and Environmental Engineering, University of Florence, 50121 Florence, ItalyDevelopments in the field of artificial intelligence have made great strides in the field of automatic semantic segmentation, both in the 2D (image) and 3D spaces. Within the context of 3D recording technology it has also seen application in several areas, most notably in creating semantically rich point clouds which is usually performed manually. In this paper, we propose the introduction of deep learning-based semantic image segmentation into the photogrammetric 3D reconstruction and classification workflow. The main objective is to be able to introduce semantic classification at the beginning of the classical photogrammetric workflow in order to automatically create classified dense point clouds by the end of the said workflow. In this regard, automatic image masking depending on pre-determined classes were performed using a previously trained neural network. The image masks were then employed during dense image matching in order to constraint the process into the respective classes, thus automatically creating semantically classified point clouds as the final output. Results show that the developed method is promising, with automation of the whole process feasible from input (images) to output (labelled point clouds). Quantitative assessment gave good results for specific classes e.g., building facades and windows, with IoU scores of 0.79 and 0.77 respectively.https://www.mdpi.com/1424-8220/22/3/966photogrammetrysemantic segmentationdeep learningautomationdense matchingpoint cloud |
| spellingShingle | Arnadi Murtiyoso Eugenio Pellis Pierre Grussenmeyer Tania Landes Andrea Masiero Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry Sensors photogrammetry semantic segmentation deep learning automation dense matching point cloud |
| title | Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry |
| title_full | Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry |
| title_fullStr | Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry |
| title_full_unstemmed | Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry |
| title_short | Towards Semantic Photogrammetry: Generating Semantically Rich Point Clouds from Architectural Close-Range Photogrammetry |
| title_sort | towards semantic photogrammetry generating semantically rich point clouds from architectural close range photogrammetry |
| topic | photogrammetry semantic segmentation deep learning automation dense matching point cloud |
| url | https://www.mdpi.com/1424-8220/22/3/966 |
| work_keys_str_mv | AT arnadimurtiyoso towardssemanticphotogrammetrygeneratingsemanticallyrichpointcloudsfromarchitecturalcloserangephotogrammetry AT eugeniopellis towardssemanticphotogrammetrygeneratingsemanticallyrichpointcloudsfromarchitecturalcloserangephotogrammetry AT pierregrussenmeyer towardssemanticphotogrammetrygeneratingsemanticallyrichpointcloudsfromarchitecturalcloserangephotogrammetry AT tanialandes towardssemanticphotogrammetrygeneratingsemanticallyrichpointcloudsfromarchitecturalcloserangephotogrammetry AT andreamasiero towardssemanticphotogrammetrygeneratingsemanticallyrichpointcloudsfromarchitecturalcloserangephotogrammetry |