A SELF-SUPERVISED KEYPOINT DETECTION NETWORK FOR MULTIMODAL REMOTE SENSING IMAGES
Currently, multimodal remote sensing images have complex geometric and radiometric distortions, which are beyond the reach of classical hand-crafted feature-based matching. Although keypoint matching methods have been developed in recent decades, most manual and deep learning-based techniques cannot...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2022-05-01
|
| Series: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| Online Access: | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2022/609/2022/isprs-archives-XLIII-B2-2022-609-2022.pdf |
| _version_ | 1818241713147215872 |
|---|---|
| author | L. Li L. Han H. Cao M. Liu |
| author_facet | L. Li L. Han H. Cao M. Liu |
| author_sort | L. Li |
| collection | DOAJ |
| description | Currently, multimodal remote sensing images have complex geometric and radiometric distortions, which are beyond the reach of classical hand-crafted feature-based matching. Although keypoint matching methods have been developed in recent decades, most manual and deep learning-based techniques cannot effectively extract highly repeatable keypoints. To address that, we design a Siamese network with self-supervised training to generate similar keypoint feature maps between multimodal images, and detect highly repeatable keypoints by computing local spatial- and channel-domain peaks of the feature maps. We exploit the confidence level of keypoints to enable the detection network to evaluate potential keypoints with end-to-end trainability. Unlike most trainable detectors, it does not require the generation of pseudo-ground truth points. In the experiments, the proposed method is evaluated using various SAR and optical images covering different scenes. The results prove its superior keypoint detection performance compared with current state-of-art matching methods based on keypoints. |
| first_indexed | 2024-12-12T13:33:42Z |
| format | Article |
| id | doaj.art-d70ed513c6524f5aad019950303e9583 |
| institution | Directory Open Access Journal |
| issn | 1682-1750 2194-9034 |
| language | English |
| last_indexed | 2024-12-12T13:33:42Z |
| publishDate | 2022-05-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| spelling | doaj.art-d70ed513c6524f5aad019950303e95832022-12-22T00:23:00ZengCopernicus PublicationsThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences1682-17502194-90342022-05-01XLIII-B2-202260961510.5194/isprs-archives-XLIII-B2-2022-609-2022A SELF-SUPERVISED KEYPOINT DETECTION NETWORK FOR MULTIMODAL REMOTE SENSING IMAGESL. Li0L. Han1H. Cao2M. Liu3College of Geological Engineering and Geomatics, Chang’an University, Xi’an 710064, ChinaSchool of Land Engineering, Chang’an University, Xi’an 710064, ChinaCollege of Geological Engineering and Geomatics, Chang’an University, Xi’an 710064, ChinaSchool of Land Engineering, Chang’an University, Xi’an 710064, ChinaCurrently, multimodal remote sensing images have complex geometric and radiometric distortions, which are beyond the reach of classical hand-crafted feature-based matching. Although keypoint matching methods have been developed in recent decades, most manual and deep learning-based techniques cannot effectively extract highly repeatable keypoints. To address that, we design a Siamese network with self-supervised training to generate similar keypoint feature maps between multimodal images, and detect highly repeatable keypoints by computing local spatial- and channel-domain peaks of the feature maps. We exploit the confidence level of keypoints to enable the detection network to evaluate potential keypoints with end-to-end trainability. Unlike most trainable detectors, it does not require the generation of pseudo-ground truth points. In the experiments, the proposed method is evaluated using various SAR and optical images covering different scenes. The results prove its superior keypoint detection performance compared with current state-of-art matching methods based on keypoints.https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2022/609/2022/isprs-archives-XLIII-B2-2022-609-2022.pdf |
| spellingShingle | L. Li L. Han H. Cao M. Liu A SELF-SUPERVISED KEYPOINT DETECTION NETWORK FOR MULTIMODAL REMOTE SENSING IMAGES The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences |
| title | A SELF-SUPERVISED KEYPOINT DETECTION NETWORK FOR MULTIMODAL REMOTE SENSING IMAGES |
| title_full | A SELF-SUPERVISED KEYPOINT DETECTION NETWORK FOR MULTIMODAL REMOTE SENSING IMAGES |
| title_fullStr | A SELF-SUPERVISED KEYPOINT DETECTION NETWORK FOR MULTIMODAL REMOTE SENSING IMAGES |
| title_full_unstemmed | A SELF-SUPERVISED KEYPOINT DETECTION NETWORK FOR MULTIMODAL REMOTE SENSING IMAGES |
| title_short | A SELF-SUPERVISED KEYPOINT DETECTION NETWORK FOR MULTIMODAL REMOTE SENSING IMAGES |
| title_sort | self supervised keypoint detection network for multimodal remote sensing images |
| url | https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2022/609/2022/isprs-archives-XLIII-B2-2022-609-2022.pdf |
| work_keys_str_mv | AT lli aselfsupervisedkeypointdetectionnetworkformultimodalremotesensingimages AT lhan aselfsupervisedkeypointdetectionnetworkformultimodalremotesensingimages AT hcao aselfsupervisedkeypointdetectionnetworkformultimodalremotesensingimages AT mliu aselfsupervisedkeypointdetectionnetworkformultimodalremotesensingimages AT lli selfsupervisedkeypointdetectionnetworkformultimodalremotesensingimages AT lhan selfsupervisedkeypointdetectionnetworkformultimodalremotesensingimages AT hcao selfsupervisedkeypointdetectionnetworkformultimodalremotesensingimages AT mliu selfsupervisedkeypointdetectionnetworkformultimodalremotesensingimages |