Multi-modal image matching to colorize a SLAM based point cloud with arbitrary data from a thermal camera
Thermal mapping of buildings can be one approach to assess the insulation, which is important in regard to upgrade buildings to increase energy efficiency and for climate change adaptation. Personal laser scanning (PLS) is a fast and flexible option that has become increasingly popular to efficientl...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-08-01
|
| Series: | ISPRS Open Journal of Photogrammetry and Remote Sensing |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2667393223000121 |
| _version_ | 1797660440245829632 |
|---|---|
| author | Melanie Elias Alexandra Weitkamp Anette Eltner |
| author_facet | Melanie Elias Alexandra Weitkamp Anette Eltner |
| author_sort | Melanie Elias |
| collection | DOAJ |
| description | Thermal mapping of buildings can be one approach to assess the insulation, which is important in regard to upgrade buildings to increase energy efficiency and for climate change adaptation. Personal laser scanning (PLS) is a fast and flexible option that has become increasingly popular to efficiently map building facades. However, some measurement systems do not include sufficient colorization of the point cloud. In order to detect, map and reference any damages to building facades, it is of great interest to transfer images from RGB and thermal infrared (TIR) cameras to the point cloud. This study aims to answer the research question if a flexible tool can be developed, which enable such measurements with high spatial resolution and flexibility. Therefore, an image-to-geometry registration approach for rendered point clouds is combined with a deep learning (DL)-based image feature matcher to estimate the camera pose of arbitrary images in relation to the geometry, i.e. the point cloud, to map color information. We developed a research design for multi-modal image matching to investigate the alignment of RGB and TIR camera images to a PLS point cloud with intensity information using calibrated and un-calibrated images. The accuracies of the estimated pose parameters reveal the best performance of the registration for pre-calibrated, i.e. undistorted, RGB camera images. The alignment of un-calibrated RGB and TIR camera images to a point cloud is possible if sufficient and well-distributed 2D-3D feature matches between image and point cloud are available. Our workflow enables the colorization of point clouds with high accuracy using images with very different radiometric characteristics and image resolutions. Only a rough approximation of the camera pose is required and hence the approach reliefs strict sensor synchronization requirements. |
| first_indexed | 2024-03-11T18:29:54Z |
| format | Article |
| id | doaj.art-5096e1da854d415e825f1d262497d418 |
| institution | Directory Open Access Journal |
| issn | 2667-3932 |
| language | English |
| last_indexed | 2024-03-11T18:29:54Z |
| publishDate | 2023-08-01 |
| publisher | Elsevier |
| record_format | Article |
| series | ISPRS Open Journal of Photogrammetry and Remote Sensing |
| spelling | doaj.art-5096e1da854d415e825f1d262497d4182023-10-13T11:06:13ZengElsevierISPRS Open Journal of Photogrammetry and Remote Sensing2667-39322023-08-019100041Multi-modal image matching to colorize a SLAM based point cloud with arbitrary data from a thermal cameraMelanie Elias0Alexandra Weitkamp1Anette Eltner2Institute of Photogrammetry & Remote Sensing, 01062, TU, Dresden, GermanyGeodetic Institute, 01062, TU, Dresden, GermanyInstitute of Photogrammetry & Remote Sensing, 01062, TU, Dresden, Germany; Corresponding author.Thermal mapping of buildings can be one approach to assess the insulation, which is important in regard to upgrade buildings to increase energy efficiency and for climate change adaptation. Personal laser scanning (PLS) is a fast and flexible option that has become increasingly popular to efficiently map building facades. However, some measurement systems do not include sufficient colorization of the point cloud. In order to detect, map and reference any damages to building facades, it is of great interest to transfer images from RGB and thermal infrared (TIR) cameras to the point cloud. This study aims to answer the research question if a flexible tool can be developed, which enable such measurements with high spatial resolution and flexibility. Therefore, an image-to-geometry registration approach for rendered point clouds is combined with a deep learning (DL)-based image feature matcher to estimate the camera pose of arbitrary images in relation to the geometry, i.e. the point cloud, to map color information. We developed a research design for multi-modal image matching to investigate the alignment of RGB and TIR camera images to a PLS point cloud with intensity information using calibrated and un-calibrated images. The accuracies of the estimated pose parameters reveal the best performance of the registration for pre-calibrated, i.e. undistorted, RGB camera images. The alignment of un-calibrated RGB and TIR camera images to a point cloud is possible if sufficient and well-distributed 2D-3D feature matches between image and point cloud are available. Our workflow enables the colorization of point clouds with high accuracy using images with very different radiometric characteristics and image resolutions. Only a rough approximation of the camera pose is required and hence the approach reliefs strict sensor synchronization requirements.http://www.sciencedirect.com/science/article/pii/S2667393223000121Thermal infrared (TIR) cameraHand-held LiDARUrban mappingDeep learningScene rendering |
| spellingShingle | Melanie Elias Alexandra Weitkamp Anette Eltner Multi-modal image matching to colorize a SLAM based point cloud with arbitrary data from a thermal camera ISPRS Open Journal of Photogrammetry and Remote Sensing Thermal infrared (TIR) camera Hand-held LiDAR Urban mapping Deep learning Scene rendering |
| title | Multi-modal image matching to colorize a SLAM based point cloud with arbitrary data from a thermal camera |
| title_full | Multi-modal image matching to colorize a SLAM based point cloud with arbitrary data from a thermal camera |
| title_fullStr | Multi-modal image matching to colorize a SLAM based point cloud with arbitrary data from a thermal camera |
| title_full_unstemmed | Multi-modal image matching to colorize a SLAM based point cloud with arbitrary data from a thermal camera |
| title_short | Multi-modal image matching to colorize a SLAM based point cloud with arbitrary data from a thermal camera |
| title_sort | multi modal image matching to colorize a slam based point cloud with arbitrary data from a thermal camera |
| topic | Thermal infrared (TIR) camera Hand-held LiDAR Urban mapping Deep learning Scene rendering |
| url | http://www.sciencedirect.com/science/article/pii/S2667393223000121 |
| work_keys_str_mv | AT melanieelias multimodalimagematchingtocolorizeaslambasedpointcloudwitharbitrarydatafromathermalcamera AT alexandraweitkamp multimodalimagematchingtocolorizeaslambasedpointcloudwitharbitrarydatafromathermalcamera AT anetteeltner multimodalimagematchingtocolorizeaslambasedpointcloudwitharbitrarydatafromathermalcamera |