Attention-Based Matching Approach for Heterogeneous Remote Sensing Images
Heterogeneous images acquired from various platforms and sensors provide complementary information. However, to use that information in applications such as image fusion and change detection, accurate image matching is essential to further process and analyze these heterogeneous images, especially i...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/1/163 |
| _version_ | 1797439873722875904 |
|---|---|
| author | Huitai Hou Chaozhen Lan Qing Xu Liang Lv Xin Xiong Fushan Yao Longhao Wang |
| author_facet | Huitai Hou Chaozhen Lan Qing Xu Liang Lv Xin Xiong Fushan Yao Longhao Wang |
| author_sort | Huitai Hou |
| collection | DOAJ |
| description | Heterogeneous images acquired from various platforms and sensors provide complementary information. However, to use that information in applications such as image fusion and change detection, accurate image matching is essential to further process and analyze these heterogeneous images, especially if they have significant differences in radiation and geometric characteristics. Therefore, matching heterogeneous remote sensing images is challenging. To address this issue, we propose a feature point matching method named Cross and Self Attentional Matcher (CSAM) based on Attention mechanisms (algorithms) that have been extensively used in various computer vision-based applications. Specifically, CSAM alternatively uses self-Attention and cross-Attention on the two matching images to exploit feature point location and context information. Then, the feature descriptor is further aggregated to assist CSAM in creating matching point pairs while removing the false matching points. To further improve the training efficiency of CSAM, this paper establishes a new training dataset of heterogeneous images, including 1,000,000 generated image pairs. Extensive experiments indicate that CSAM outperforms the existing feature extraction and matching methods, including SIFT, RIFT, CFOG, NNDR, FSC, GMS, OA-Net, and Superglue, attaining an average precision and processing time of 81.29% and 0.13 s. In addition to higher matching performance and computational efficiency, CSAM has better generalization ability for multimodal image matching and registration tasks. |
| first_indexed | 2024-03-09T11:59:03Z |
| format | Article |
| id | doaj.art-d4c951b3b39d4c69a034925f0d47010f |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T11:59:03Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-d4c951b3b39d4c69a034925f0d47010f2023-11-30T23:05:57ZengMDPI AGRemote Sensing2072-42922022-12-0115116310.3390/rs15010163Attention-Based Matching Approach for Heterogeneous Remote Sensing ImagesHuitai Hou0Chaozhen Lan1Qing Xu2Liang Lv3Xin Xiong4Fushan Yao5Longhao Wang6Institute of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaInstitute of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaInstitute of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaInstitute of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaInstitute of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaInstitute of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaInstitute of Geospatial Information, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, ChinaHeterogeneous images acquired from various platforms and sensors provide complementary information. However, to use that information in applications such as image fusion and change detection, accurate image matching is essential to further process and analyze these heterogeneous images, especially if they have significant differences in radiation and geometric characteristics. Therefore, matching heterogeneous remote sensing images is challenging. To address this issue, we propose a feature point matching method named Cross and Self Attentional Matcher (CSAM) based on Attention mechanisms (algorithms) that have been extensively used in various computer vision-based applications. Specifically, CSAM alternatively uses self-Attention and cross-Attention on the two matching images to exploit feature point location and context information. Then, the feature descriptor is further aggregated to assist CSAM in creating matching point pairs while removing the false matching points. To further improve the training efficiency of CSAM, this paper establishes a new training dataset of heterogeneous images, including 1,000,000 generated image pairs. Extensive experiments indicate that CSAM outperforms the existing feature extraction and matching methods, including SIFT, RIFT, CFOG, NNDR, FSC, GMS, OA-Net, and Superglue, attaining an average precision and processing time of 81.29% and 0.13 s. In addition to higher matching performance and computational efficiency, CSAM has better generalization ability for multimodal image matching and registration tasks.https://www.mdpi.com/2072-4292/15/1/163heterogeneous remote sensing imagesimage matchingself-attentioncross-attentiondeep learning |
| spellingShingle | Huitai Hou Chaozhen Lan Qing Xu Liang Lv Xin Xiong Fushan Yao Longhao Wang Attention-Based Matching Approach for Heterogeneous Remote Sensing Images Remote Sensing heterogeneous remote sensing images image matching self-attention cross-attention deep learning |
| title | Attention-Based Matching Approach for Heterogeneous Remote Sensing Images |
| title_full | Attention-Based Matching Approach for Heterogeneous Remote Sensing Images |
| title_fullStr | Attention-Based Matching Approach for Heterogeneous Remote Sensing Images |
| title_full_unstemmed | Attention-Based Matching Approach for Heterogeneous Remote Sensing Images |
| title_short | Attention-Based Matching Approach for Heterogeneous Remote Sensing Images |
| title_sort | attention based matching approach for heterogeneous remote sensing images |
| topic | heterogeneous remote sensing images image matching self-attention cross-attention deep learning |
| url | https://www.mdpi.com/2072-4292/15/1/163 |
| work_keys_str_mv | AT huitaihou attentionbasedmatchingapproachforheterogeneousremotesensingimages AT chaozhenlan attentionbasedmatchingapproachforheterogeneousremotesensingimages AT qingxu attentionbasedmatchingapproachforheterogeneousremotesensingimages AT lianglv attentionbasedmatchingapproachforheterogeneousremotesensingimages AT xinxiong attentionbasedmatchingapproachforheterogeneousremotesensingimages AT fushanyao attentionbasedmatchingapproachforheterogeneousremotesensingimages AT longhaowang attentionbasedmatchingapproachforheterogeneousremotesensingimages |