JOINT BUNDLE ADJUSTMENT OF THERMAL INFRA-RED AND OPTICAL IMAGES BASED ON MULTIMODAL MATCHING

Despite the fact that Thermal Infrared (TIR) cameras have been in use for decades, processing of TIR images poses a variety of challenges when compared to optical images, which are captured in the visible part of the electromagnetic spectrum. The estimation of the exterior orientation of TIR cameras...

Full description

Bibliographic Details
Main Authors: A. Sledz, C. Heipke
Format: Article
Language:English
Published: Copernicus Publications 2022-05-01
Series:The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Online Access:https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B1-2022/157/2022/isprs-archives-XLIII-B1-2022-157-2022.pdf
_version_ 1817987106795945984
author A. Sledz
C. Heipke
author_facet A. Sledz
C. Heipke
author_sort A. Sledz
collection DOAJ
description Despite the fact that Thermal Infrared (TIR) cameras have been in use for decades, processing of TIR images poses a variety of challenges when compared to optical images, which are captured in the visible part of the electromagnetic spectrum. The estimation of the exterior orientation of TIR cameras by bundle adjustment is a difficult task due to the limited geometric resolution of a TIR camera and the low image quality in terms of contrast and texture compared to optical images. Optical images have a potential to increase TIR external orientation accuracy by incorporating them into a joint bundle adjustment. However, because the modality gap between those two image types is large, classical point matching algorithms typically fail to find matches, making processing both image types in the joint bundle challenging. In order to locate matching points in both modalities, this study suggests using the Edge Histogram Descriptor (EHD) in the frequency domain representation of the images based on phase congruency. To properly allocate edges from the phase congruency, which are then employed in EHD, non-maximum suppression and hysteresis thresholding are used. Considering that both sensors are fixed rigidly to a single platform, the search region for the matching point candidate of the TIR image is determined based on stereo calibration of a thermal/optical stereo setup combined with geometric constraints. The final matching is based on the cosine distance, while RANdom SAmple Consensus (RANSAC) is used in order to eliminate outliers. The findings of this study show that using a joint bundle adjustment with optical images versus a bundle adjustment only with TIR images improves TIR image orientation, which is supported by the increased accuracy of the adjusted Ground Control Point (GCP) coordinates.
first_indexed 2024-04-14T00:17:40Z
format Article
id doaj.art-0bd99508eb2f467dbd5041cb87ed9e95
institution Directory Open Access Journal
issn 1682-1750
2194-9034
language English
last_indexed 2024-04-14T00:17:40Z
publishDate 2022-05-01
publisher Copernicus Publications
record_format Article
series The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
spelling doaj.art-0bd99508eb2f467dbd5041cb87ed9e952022-12-22T02:23:03ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342022-05-01XLIII-B1-202215716510.5194/isprs-archives-XLIII-B1-2022-157-2022JOINT BUNDLE ADJUSTMENT OF THERMAL INFRA-RED AND OPTICAL IMAGES BASED ON MULTIMODAL MATCHINGA. Sledz0C. Heipke1Institute of Photogrammetry and GeoInformation, Leibniz Universität Hannover, GermanyInstitute of Photogrammetry and GeoInformation, Leibniz Universität Hannover, GermanyDespite the fact that Thermal Infrared (TIR) cameras have been in use for decades, processing of TIR images poses a variety of challenges when compared to optical images, which are captured in the visible part of the electromagnetic spectrum. The estimation of the exterior orientation of TIR cameras by bundle adjustment is a difficult task due to the limited geometric resolution of a TIR camera and the low image quality in terms of contrast and texture compared to optical images. Optical images have a potential to increase TIR external orientation accuracy by incorporating them into a joint bundle adjustment. However, because the modality gap between those two image types is large, classical point matching algorithms typically fail to find matches, making processing both image types in the joint bundle challenging. In order to locate matching points in both modalities, this study suggests using the Edge Histogram Descriptor (EHD) in the frequency domain representation of the images based on phase congruency. To properly allocate edges from the phase congruency, which are then employed in EHD, non-maximum suppression and hysteresis thresholding are used. Considering that both sensors are fixed rigidly to a single platform, the search region for the matching point candidate of the TIR image is determined based on stereo calibration of a thermal/optical stereo setup combined with geometric constraints. The final matching is based on the cosine distance, while RANdom SAmple Consensus (RANSAC) is used in order to eliminate outliers. The findings of this study show that using a joint bundle adjustment with optical images versus a bundle adjustment only with TIR images improves TIR image orientation, which is supported by the increased accuracy of the adjusted Ground Control Point (GCP) coordinates.https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B1-2022/157/2022/isprs-archives-XLIII-B1-2022-157-2022.pdf
spellingShingle A. Sledz
C. Heipke
JOINT BUNDLE ADJUSTMENT OF THERMAL INFRA-RED AND OPTICAL IMAGES BASED ON MULTIMODAL MATCHING
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
title JOINT BUNDLE ADJUSTMENT OF THERMAL INFRA-RED AND OPTICAL IMAGES BASED ON MULTIMODAL MATCHING
title_full JOINT BUNDLE ADJUSTMENT OF THERMAL INFRA-RED AND OPTICAL IMAGES BASED ON MULTIMODAL MATCHING
title_fullStr JOINT BUNDLE ADJUSTMENT OF THERMAL INFRA-RED AND OPTICAL IMAGES BASED ON MULTIMODAL MATCHING
title_full_unstemmed JOINT BUNDLE ADJUSTMENT OF THERMAL INFRA-RED AND OPTICAL IMAGES BASED ON MULTIMODAL MATCHING
title_short JOINT BUNDLE ADJUSTMENT OF THERMAL INFRA-RED AND OPTICAL IMAGES BASED ON MULTIMODAL MATCHING
title_sort joint bundle adjustment of thermal infra red and optical images based on multimodal matching
url https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B1-2022/157/2022/isprs-archives-XLIII-B1-2022-157-2022.pdf
work_keys_str_mv AT asledz jointbundleadjustmentofthermalinfraredandopticalimagesbasedonmultimodalmatching
AT cheipke jointbundleadjustmentofthermalinfraredandopticalimagesbasedonmultimodalmatching