Efficient and Robust: A Cross-Modal Registration Deep Wavelet Learning Method for Remote Sensing Images
Deep convolutional networks are powerful for local feature learning and have shown advantages in image matching and registration. However, the significant differences between cross-modal images increase the challenge of image registration. The deep network should extract modality-invariant features...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10124986/ |
| _version_ | 1797817096893104128 |
|---|---|
| author | Dou Quan Huiyuan Wei Shuang Wang Yi Li Jocelyn Chanussot Yanhe Guo Biao Hou Licheng Jiao |
| author_facet | Dou Quan Huiyuan Wei Shuang Wang Yi Li Jocelyn Chanussot Yanhe Guo Biao Hou Licheng Jiao |
| author_sort | Dou Quan |
| collection | DOAJ |
| description | Deep convolutional networks are powerful for local feature learning and have shown advantages in image matching and registration. However, the significant differences between cross-modal images increase the challenge of image registration. The deep network should extract modality-invariant features to identify the matching samples and discriminative features to separate the nonmatching samples. The deep network can extract features invariant to the image modality changes by multiple nonlinear mapping layers. However, it does not inevitably lose rich details and affect the discrimination of features, degrading registration performances. This article proposes a novel deep wavelet learning network (DW-Net) for local feature learning. It incorporates spectral information into deep convolutional features for improving cross-modal image matching and registration. Specifically, this article aims to learn the multiresolution wavelet features through multilevel wavelet transform (WT) and the convolutional network. The cross-modal images are divided into low-frequency and high-frequency parts through WT. DW-Net can adaptively extract the shared features from the low-frequency part and useful details from the high-frequency part, which can enhance the modality invariance and discrimination of features. Additionally, the multiresolution wavelet features contain multiscale information and contribute to improving the matching accuracy. Extensive experiments demonstrate the significant advantages in terms of the accuracy and robustness of DW-Net on cross-modal remote sensing image registration. DW-Net can increase the image patch matching accuracy by 3.7% and improve image registration probability by 12.1%. Moreover, DW-Net shows strong generalization performances from low resolution to high resolution and from optical– synthetic aperture radar to other cross-modal image registration. |
| first_indexed | 2024-03-13T08:46:51Z |
| format | Article |
| id | doaj.art-3b77c662ffc14c939c5566a54f0fd54d |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-03-13T08:46:51Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-3b77c662ffc14c939c5566a54f0fd54d2023-05-29T23:00:13ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352023-01-01164739475410.1109/JSTARS.2023.327640910124986Efficient and Robust: A Cross-Modal Registration Deep Wavelet Learning Method for Remote Sensing ImagesDou Quan0https://orcid.org/0000-0001-6943-4657Huiyuan Wei1Shuang Wang2https://orcid.org/0000-0003-4940-1211Yi Li3https://orcid.org/0000-0003-4226-6635Jocelyn Chanussot4https://orcid.org/0000-0003-4817-2875Yanhe Guo5Biao Hou6Licheng Jiao7https://orcid.org/0000-0003-3354-9617Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China, Xidian University, Xi'an, ChinaKey Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China, Xidian University, Xi'an, ChinaKey Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China, Xidian University, Xi'an, ChinaSchool of Telecommunications, Xidian University, Xi'an, ChinaUniversity Grenoble Alpes, CNRS, Grenoble INP, GIPSA-lab, Grenoble, FranceKey Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China, Xidian University, Xi'an, ChinaKey Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China, Xidian University, Xi'an, ChinaKey Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China, Xidian University, Xi'an, ChinaDeep convolutional networks are powerful for local feature learning and have shown advantages in image matching and registration. However, the significant differences between cross-modal images increase the challenge of image registration. The deep network should extract modality-invariant features to identify the matching samples and discriminative features to separate the nonmatching samples. The deep network can extract features invariant to the image modality changes by multiple nonlinear mapping layers. However, it does not inevitably lose rich details and affect the discrimination of features, degrading registration performances. This article proposes a novel deep wavelet learning network (DW-Net) for local feature learning. It incorporates spectral information into deep convolutional features for improving cross-modal image matching and registration. Specifically, this article aims to learn the multiresolution wavelet features through multilevel wavelet transform (WT) and the convolutional network. The cross-modal images are divided into low-frequency and high-frequency parts through WT. DW-Net can adaptively extract the shared features from the low-frequency part and useful details from the high-frequency part, which can enhance the modality invariance and discrimination of features. Additionally, the multiresolution wavelet features contain multiscale information and contribute to improving the matching accuracy. Extensive experiments demonstrate the significant advantages in terms of the accuracy and robustness of DW-Net on cross-modal remote sensing image registration. DW-Net can increase the image patch matching accuracy by 3.7% and improve image registration probability by 12.1%. Moreover, DW-Net shows strong generalization performances from low resolution to high resolution and from optical– synthetic aperture radar to other cross-modal image registration.https://ieeexplore.ieee.org/document/10124986/Cross-modal imagesdiscriminative featuresimage registrationmodality-invariantmultiresolutionwavelet features |
| spellingShingle | Dou Quan Huiyuan Wei Shuang Wang Yi Li Jocelyn Chanussot Yanhe Guo Biao Hou Licheng Jiao Efficient and Robust: A Cross-Modal Registration Deep Wavelet Learning Method for Remote Sensing Images IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Cross-modal images discriminative features image registration modality-invariant multiresolution wavelet features |
| title | Efficient and Robust: A Cross-Modal Registration Deep Wavelet Learning Method for Remote Sensing Images |
| title_full | Efficient and Robust: A Cross-Modal Registration Deep Wavelet Learning Method for Remote Sensing Images |
| title_fullStr | Efficient and Robust: A Cross-Modal Registration Deep Wavelet Learning Method for Remote Sensing Images |
| title_full_unstemmed | Efficient and Robust: A Cross-Modal Registration Deep Wavelet Learning Method for Remote Sensing Images |
| title_short | Efficient and Robust: A Cross-Modal Registration Deep Wavelet Learning Method for Remote Sensing Images |
| title_sort | efficient and robust a cross modal registration deep wavelet learning method for remote sensing images |
| topic | Cross-modal images discriminative features image registration modality-invariant multiresolution wavelet features |
| url | https://ieeexplore.ieee.org/document/10124986/ |
| work_keys_str_mv | AT douquan efficientandrobustacrossmodalregistrationdeepwaveletlearningmethodforremotesensingimages AT huiyuanwei efficientandrobustacrossmodalregistrationdeepwaveletlearningmethodforremotesensingimages AT shuangwang efficientandrobustacrossmodalregistrationdeepwaveletlearningmethodforremotesensingimages AT yili efficientandrobustacrossmodalregistrationdeepwaveletlearningmethodforremotesensingimages AT jocelynchanussot efficientandrobustacrossmodalregistrationdeepwaveletlearningmethodforremotesensingimages AT yanheguo efficientandrobustacrossmodalregistrationdeepwaveletlearningmethodforremotesensingimages AT biaohou efficientandrobustacrossmodalregistrationdeepwaveletlearningmethodforremotesensingimages AT lichengjiao efficientandrobustacrossmodalregistrationdeepwaveletlearningmethodforremotesensingimages |