A Deep Cross-Modality Hashing Network for SAR and Optical Remote Sensing Images Retrieval
The content-based remote sensing image retrieval (CBRSIR) has recently become a hot topic due to its wide applications in analysis of remote sensing data. However, since conventional CBRSIR is unsuitable in harsh environments, this article focuses on the cross-modality CBRSIR (CM-CBRSIR) between syn...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9186284/ |
| _version_ | 1818740381008789504 |
|---|---|
| author | Wei Xiong Zhenyu Xiong Yang Zhang Yaqi Cui Xiangqi Gu |
| author_facet | Wei Xiong Zhenyu Xiong Yang Zhang Yaqi Cui Xiangqi Gu |
| author_sort | Wei Xiong |
| collection | DOAJ |
| description | The content-based remote sensing image retrieval (CBRSIR) has recently become a hot topic due to its wide applications in analysis of remote sensing data. However, since conventional CBRSIR is unsuitable in harsh environments, this article focuses on the cross-modality CBRSIR (CM-CBRSIR) between synthetic aperture radar (SAR) and optical images. Besides the large interclass and small intraclass in CBRSIR, CM-CBRSIR is limited by prominent modality discrepancy caused by different imaging mechanisms. To address this limitation, this study proposes a deep cross-modality hashing network. First, we transform optical images with three channels into four different types of single-channel images to increase diversity of the training modalities. This helps the network to mainly focus on extracting the contour and texture shared features and makes it less sensitive to color information for images across modalities. Second, we combine any type of randomly selected transformed images and its corresponding SAR or optical images to form image pairs that are fed into the networks. The training strategy, with paired image data, eliminates the large cross-modality variations caused by different modalities. Finally, the triplet loss, in combination with the hash function, helps the modal to extract the discriminative features of images and upgrade the retrieval efficiency. To further evaluate the proposed modality, we construct a SAR-optical dual-modality remote sensing image dataset containing 12 categories. Experimental results demonstrate the superiority of the proposed method with regards to efficiency and generality. |
| first_indexed | 2024-12-18T01:39:49Z |
| format | Article |
| id | doaj.art-35789ac6d900465f92f48bc668f19847 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-12-18T01:39:49Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-35789ac6d900465f92f48bc668f198472022-12-21T21:25:22ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352020-01-01135284529610.1109/JSTARS.2020.30213909186284A Deep Cross-Modality Hashing Network for SAR and Optical Remote Sensing Images RetrievalWei Xiong0Zhenyu Xiong1https://orcid.org/0000-0002-1277-7875Yang Zhang2Yaqi Cui3Xiangqi Gu4Research Institute of information Fusion, Naval Aviation University, Yantai, ChinaResearch Institute of information Fusion, Naval Aviation University, Yantai, ChinaResearch Institute of information Fusion, Naval Aviation University, Yantai, ChinaResearch Institute of information Fusion, Naval Aviation University, Yantai, ChinaResearch Institute of information Fusion, Naval Aviation University, Yantai, ChinaThe content-based remote sensing image retrieval (CBRSIR) has recently become a hot topic due to its wide applications in analysis of remote sensing data. However, since conventional CBRSIR is unsuitable in harsh environments, this article focuses on the cross-modality CBRSIR (CM-CBRSIR) between synthetic aperture radar (SAR) and optical images. Besides the large interclass and small intraclass in CBRSIR, CM-CBRSIR is limited by prominent modality discrepancy caused by different imaging mechanisms. To address this limitation, this study proposes a deep cross-modality hashing network. First, we transform optical images with three channels into four different types of single-channel images to increase diversity of the training modalities. This helps the network to mainly focus on extracting the contour and texture shared features and makes it less sensitive to color information for images across modalities. Second, we combine any type of randomly selected transformed images and its corresponding SAR or optical images to form image pairs that are fed into the networks. The training strategy, with paired image data, eliminates the large cross-modality variations caused by different modalities. Finally, the triplet loss, in combination with the hash function, helps the modal to extract the discriminative features of images and upgrade the retrieval efficiency. To further evaluate the proposed modality, we construct a SAR-optical dual-modality remote sensing image dataset containing 12 categories. Experimental results demonstrate the superiority of the proposed method with regards to efficiency and generality.https://ieeexplore.ieee.org/document/9186284/Cross-modality content-based remote sensing image retrieval (CM-CBRSIR)deep cross-modality hashing network (DCMHN)modality discrepancysynthetic aperture radar (SAR)-optical dual-modality remote sensing image dataset (SODMRSID) |
| spellingShingle | Wei Xiong Zhenyu Xiong Yang Zhang Yaqi Cui Xiangqi Gu A Deep Cross-Modality Hashing Network for SAR and Optical Remote Sensing Images Retrieval IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Cross-modality content-based remote sensing image retrieval (CM-CBRSIR) deep cross-modality hashing network (DCMHN) modality discrepancy synthetic aperture radar (SAR)-optical dual-modality remote sensing image dataset (SODMRSID) |
| title | A Deep Cross-Modality Hashing Network for SAR and Optical Remote Sensing Images Retrieval |
| title_full | A Deep Cross-Modality Hashing Network for SAR and Optical Remote Sensing Images Retrieval |
| title_fullStr | A Deep Cross-Modality Hashing Network for SAR and Optical Remote Sensing Images Retrieval |
| title_full_unstemmed | A Deep Cross-Modality Hashing Network for SAR and Optical Remote Sensing Images Retrieval |
| title_short | A Deep Cross-Modality Hashing Network for SAR and Optical Remote Sensing Images Retrieval |
| title_sort | deep cross modality hashing network for sar and optical remote sensing images retrieval |
| topic | Cross-modality content-based remote sensing image retrieval (CM-CBRSIR) deep cross-modality hashing network (DCMHN) modality discrepancy synthetic aperture radar (SAR)-optical dual-modality remote sensing image dataset (SODMRSID) |
| url | https://ieeexplore.ieee.org/document/9186284/ |
| work_keys_str_mv | AT weixiong adeepcrossmodalityhashingnetworkforsarandopticalremotesensingimagesretrieval AT zhenyuxiong adeepcrossmodalityhashingnetworkforsarandopticalremotesensingimagesretrieval AT yangzhang adeepcrossmodalityhashingnetworkforsarandopticalremotesensingimagesretrieval AT yaqicui adeepcrossmodalityhashingnetworkforsarandopticalremotesensingimagesretrieval AT xiangqigu adeepcrossmodalityhashingnetworkforsarandopticalremotesensingimagesretrieval AT weixiong deepcrossmodalityhashingnetworkforsarandopticalremotesensingimagesretrieval AT zhenyuxiong deepcrossmodalityhashingnetworkforsarandopticalremotesensingimagesretrieval AT yangzhang deepcrossmodalityhashingnetworkforsarandopticalremotesensingimagesretrieval AT yaqicui deepcrossmodalityhashingnetworkforsarandopticalremotesensingimagesretrieval AT xiangqigu deepcrossmodalityhashingnetworkforsarandopticalremotesensingimagesretrieval |