IML-Net: A Framework for Cross-View Geo-Localization with Multi-Domain Remote Sensing Data
Cross-view geolocation is a valuable yet challenging task. In practical applications, the images targeted by cross-view geolocation technology encompass multi-domain remote sensing images, including those from different platforms (e.g., drone cameras and satellites), different perspectives (e.g., na...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/16/7/1249 |
| _version_ | 1797212003677241344 |
|---|---|
| author | Yiming Yan Mengyuan Wang Nan Su Wei Hou Chunhui Zhao Wenxuan Wang |
| author_facet | Yiming Yan Mengyuan Wang Nan Su Wei Hou Chunhui Zhao Wenxuan Wang |
| author_sort | Yiming Yan |
| collection | DOAJ |
| description | Cross-view geolocation is a valuable yet challenging task. In practical applications, the images targeted by cross-view geolocation technology encompass multi-domain remote sensing images, including those from different platforms (e.g., drone cameras and satellites), different perspectives (e.g., nadir and oblique), and different temporal conditions (e.g., various seasons and weather conditions). Based on the characteristics of these images, we have designed an effective framework, Image Reconstruction and Multi-Unit Mutual Learning Net (IML-Net), for accomplishing cross-view geolocation tasks. By incorporating a deconvolutional network into the architecture to reconstruct images, we can better bridge the differences in remote sensing image features across different domains. This enables the mapping of target images from different platforms and perspectives into a shared latent space representation, obtaining more discriminative feature descriptors. The process enhances the robustness of feature extraction for locating targets across a wide range of perspectives. To improve the network’s performance, we introduce attention regions learned from different units as augmented data during the training process. For the current cross-view geolocation datasets, the use of large-scale datasets is limited due to high costs and privacy concerns, leading to the prevalent use of simulated data. However, real data allow the network to learn more generalizable features. To make the model more robust and stable, we collected two groups of multi-domain datasets from the Zurich and Harbin regions, incorporating real data into the cross-view geolocation task to construct the ZHcity750 Dataset. Our framework is evaluated on the cross-domain ZHcity750 Dataset, which shows competitive results compared to state-of-the-art methods. |
| first_indexed | 2024-04-24T10:35:29Z |
| format | Article |
| id | doaj.art-459c465993954c73a695d5d04756e3a5 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-04-24T10:35:29Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-459c465993954c73a695d5d04756e3a52024-04-12T13:25:45ZengMDPI AGRemote Sensing2072-42922024-03-01167124910.3390/rs16071249IML-Net: A Framework for Cross-View Geo-Localization with Multi-Domain Remote Sensing DataYiming Yan0Mengyuan Wang1Nan Su2Wei Hou3Chunhui Zhao4Wenxuan Wang5College of Information and Communication Engineering, Harbin Engineering University, Harbin 150009, ChinaCollege of Information and Communication Engineering, Harbin Engineering University, Harbin 150009, ChinaCollege of Information and Communication Engineering, Harbin Engineering University, Harbin 150009, ChinaHarbin Aerospace Star Data System Science and Technology Co., Ltd., Harbin 150028, ChinaCollege of Information and Communication Engineering, Harbin Engineering University, Harbin 150009, ChinaCollege of Information and Communication Engineering, Harbin Engineering University, Harbin 150009, ChinaCross-view geolocation is a valuable yet challenging task. In practical applications, the images targeted by cross-view geolocation technology encompass multi-domain remote sensing images, including those from different platforms (e.g., drone cameras and satellites), different perspectives (e.g., nadir and oblique), and different temporal conditions (e.g., various seasons and weather conditions). Based on the characteristics of these images, we have designed an effective framework, Image Reconstruction and Multi-Unit Mutual Learning Net (IML-Net), for accomplishing cross-view geolocation tasks. By incorporating a deconvolutional network into the architecture to reconstruct images, we can better bridge the differences in remote sensing image features across different domains. This enables the mapping of target images from different platforms and perspectives into a shared latent space representation, obtaining more discriminative feature descriptors. The process enhances the robustness of feature extraction for locating targets across a wide range of perspectives. To improve the network’s performance, we introduce attention regions learned from different units as augmented data during the training process. For the current cross-view geolocation datasets, the use of large-scale datasets is limited due to high costs and privacy concerns, leading to the prevalent use of simulated data. However, real data allow the network to learn more generalizable features. To make the model more robust and stable, we collected two groups of multi-domain datasets from the Zurich and Harbin regions, incorporating real data into the cross-view geolocation task to construct the ZHcity750 Dataset. Our framework is evaluated on the cross-domain ZHcity750 Dataset, which shows competitive results compared to state-of-the-art methods.https://www.mdpi.com/2072-4292/16/7/1249geo-localizationmulti-domainIML-NetZHcity750 |
| spellingShingle | Yiming Yan Mengyuan Wang Nan Su Wei Hou Chunhui Zhao Wenxuan Wang IML-Net: A Framework for Cross-View Geo-Localization with Multi-Domain Remote Sensing Data Remote Sensing geo-localization multi-domain IML-Net ZHcity750 |
| title | IML-Net: A Framework for Cross-View Geo-Localization with Multi-Domain Remote Sensing Data |
| title_full | IML-Net: A Framework for Cross-View Geo-Localization with Multi-Domain Remote Sensing Data |
| title_fullStr | IML-Net: A Framework for Cross-View Geo-Localization with Multi-Domain Remote Sensing Data |
| title_full_unstemmed | IML-Net: A Framework for Cross-View Geo-Localization with Multi-Domain Remote Sensing Data |
| title_short | IML-Net: A Framework for Cross-View Geo-Localization with Multi-Domain Remote Sensing Data |
| title_sort | iml net a framework for cross view geo localization with multi domain remote sensing data |
| topic | geo-localization multi-domain IML-Net ZHcity750 |
| url | https://www.mdpi.com/2072-4292/16/7/1249 |
| work_keys_str_mv | AT yimingyan imlnetaframeworkforcrossviewgeolocalizationwithmultidomainremotesensingdata AT mengyuanwang imlnetaframeworkforcrossviewgeolocalizationwithmultidomainremotesensingdata AT nansu imlnetaframeworkforcrossviewgeolocalizationwithmultidomainremotesensingdata AT weihou imlnetaframeworkforcrossviewgeolocalizationwithmultidomainremotesensingdata AT chunhuizhao imlnetaframeworkforcrossviewgeolocalizationwithmultidomainremotesensingdata AT wenxuanwang imlnetaframeworkforcrossviewgeolocalizationwithmultidomainremotesensingdata |