Self-Supervised Pre-Training with Bridge Neural Network for SAR-Optical Matching
Due to the vast geometric and radiometric differences between SAR and optical images, SAR-optical image matching remains an intractable challenge. Despite the fact that the deep learning-based matching model has achieved great success, SAR feature embedding ability is not fully explored yet because...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/12/2749 |
| _version_ | 1797482722079277056 |
|---|---|
| author | Lixin Qian Xiaochun Liu Meiyu Huang Xueshuang Xiang |
| author_facet | Lixin Qian Xiaochun Liu Meiyu Huang Xueshuang Xiang |
| author_sort | Lixin Qian |
| collection | DOAJ |
| description | Due to the vast geometric and radiometric differences between SAR and optical images, SAR-optical image matching remains an intractable challenge. Despite the fact that the deep learning-based matching model has achieved great success, SAR feature embedding ability is not fully explored yet because of the lack of well-designed pre-training techniques. In this paper, we propose to employ the self-supervised learning method in the SAR-optical matching framework, in order to serve as a pre-training strategy for improving the representation learning ability of SAR images as well as optical images. We first use a state-of-the-art self-supervised learning method, Momentum Contrast (MoCo), to pre-train an optical feature encoder and an SAR feature encoder separately. Then, the pre-trained encoders are transferred to an advanced common representation learning model, Bridge Neural Network (BNN), to project the SAR and optical images into a more distinguishable common feature representation subspace, which leads to a high multi-modal image matching result. Experimental results on three SAR-optical matching benchmark datasets show that our proposed MoCo pre-training method achieves a high matching accuracy up to 0.873 even for the complex QXS-SAROPT SAR-optical matching dataset. BNN pre-trained with MoCo outperforms BNN with the most commonly used ImageNet pre-training, and achieves at most 4.4% gains in matching accuracy. |
| first_indexed | 2024-03-09T22:37:31Z |
| format | Article |
| id | doaj.art-31cc14bc5c8641bda3769001c2887616 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T22:37:31Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-31cc14bc5c8641bda3769001c28876162023-11-23T18:46:11ZengMDPI AGRemote Sensing2072-42922022-06-011412274910.3390/rs14122749Self-Supervised Pre-Training with Bridge Neural Network for SAR-Optical MatchingLixin Qian0Xiaochun Liu1Meiyu Huang2Xueshuang Xiang3School of Mathematics and Statistics, Wuhan University, Wuchang District, Wuhan 430072, ChinaSchool of Mathematics and Statistics, Wuhan University, Wuchang District, Wuhan 430072, ChinaQian Xuesen Laboratory of Space Technology, China Academy of Space Technology, Haidian District, Beijing 100086, ChinaQian Xuesen Laboratory of Space Technology, China Academy of Space Technology, Haidian District, Beijing 100086, ChinaDue to the vast geometric and radiometric differences between SAR and optical images, SAR-optical image matching remains an intractable challenge. Despite the fact that the deep learning-based matching model has achieved great success, SAR feature embedding ability is not fully explored yet because of the lack of well-designed pre-training techniques. In this paper, we propose to employ the self-supervised learning method in the SAR-optical matching framework, in order to serve as a pre-training strategy for improving the representation learning ability of SAR images as well as optical images. We first use a state-of-the-art self-supervised learning method, Momentum Contrast (MoCo), to pre-train an optical feature encoder and an SAR feature encoder separately. Then, the pre-trained encoders are transferred to an advanced common representation learning model, Bridge Neural Network (BNN), to project the SAR and optical images into a more distinguishable common feature representation subspace, which leads to a high multi-modal image matching result. Experimental results on three SAR-optical matching benchmark datasets show that our proposed MoCo pre-training method achieves a high matching accuracy up to 0.873 even for the complex QXS-SAROPT SAR-optical matching dataset. BNN pre-trained with MoCo outperforms BNN with the most commonly used ImageNet pre-training, and achieves at most 4.4% gains in matching accuracy.https://www.mdpi.com/2072-4292/14/12/2749SAR-optical fusionimage matchingself-supervised learningrepresentation learning |
| spellingShingle | Lixin Qian Xiaochun Liu Meiyu Huang Xueshuang Xiang Self-Supervised Pre-Training with Bridge Neural Network for SAR-Optical Matching Remote Sensing SAR-optical fusion image matching self-supervised learning representation learning |
| title | Self-Supervised Pre-Training with Bridge Neural Network for SAR-Optical Matching |
| title_full | Self-Supervised Pre-Training with Bridge Neural Network for SAR-Optical Matching |
| title_fullStr | Self-Supervised Pre-Training with Bridge Neural Network for SAR-Optical Matching |
| title_full_unstemmed | Self-Supervised Pre-Training with Bridge Neural Network for SAR-Optical Matching |
| title_short | Self-Supervised Pre-Training with Bridge Neural Network for SAR-Optical Matching |
| title_sort | self supervised pre training with bridge neural network for sar optical matching |
| topic | SAR-optical fusion image matching self-supervised learning representation learning |
| url | https://www.mdpi.com/2072-4292/14/12/2749 |
| work_keys_str_mv | AT lixinqian selfsupervisedpretrainingwithbridgeneuralnetworkforsaropticalmatching AT xiaochunliu selfsupervisedpretrainingwithbridgeneuralnetworkforsaropticalmatching AT meiyuhuang selfsupervisedpretrainingwithbridgeneuralnetworkforsaropticalmatching AT xueshuangxiang selfsupervisedpretrainingwithbridgeneuralnetworkforsaropticalmatching |