EGMT-CD: Edge-Guided Multimodal Transformers Change Detection from Satellite and Aerial Images
Change detection from heterogeneous satellite and aerial images plays a progressively important role in many fields, including disaster assessment, urban construction, and land use monitoring. Currently, researchers have mainly devoted their attention to change detection using homologous image pairs...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-12-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/16/1/86 |
| _version_ | 1797358181893013504 |
|---|---|
| author | Yunfan Xiang Xiangyu Tian Yue Xu Xiaokun Guan Zhengchao Chen |
| author_facet | Yunfan Xiang Xiangyu Tian Yue Xu Xiaokun Guan Zhengchao Chen |
| author_sort | Yunfan Xiang |
| collection | DOAJ |
| description | Change detection from heterogeneous satellite and aerial images plays a progressively important role in many fields, including disaster assessment, urban construction, and land use monitoring. Currently, researchers have mainly devoted their attention to change detection using homologous image pairs and achieved many remarkable results. It is sometimes necessary to use heterogeneous images for change detection in practical scenarios due to missing images, emergency situations, and cloud and fog occlusion. However, heterogeneous change detection still faces great challenges, especially using satellite and aerial images. The main challenges in satellite and aerial image change detection are related to the resolution gap and blurred edge. Previous studies used interpolation or shallow feature alignment before traditional homologous change detection methods, which ignored the high-level feature interaction and edge information. Therefore, we propose a new heterogeneous change detection model based on multimodal transformers combined with edge guidance. In order to alleviate the resolution gap between satellite and aerial images, we design an improved spatially aligned transformer (SP-T) with a sub-pixel module to align the satellite features to the same size of the aerial ones supervised by a token loss. Moreover, we introduce an edge detection branch to guide change features using the object edge with an auxiliary edge-change loss. Finally, we conduct considerable experiments to verify the effectiveness and superiority of our proposed model (EGMT-CD) on a new satellite–aerial heterogeneous change dataset, named SACD. The experiments show that our method (EGMT-CD) outperforms many previously superior change detection methods and fully demonstrates its potential in heterogeneous change detection from satellite–aerial images. |
| first_indexed | 2024-03-08T14:58:12Z |
| format | Article |
| id | doaj.art-37d1e63517834480b3cb60037f0d1ec9 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-08T14:58:12Z |
| publishDate | 2023-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-37d1e63517834480b3cb60037f0d1ec92024-01-10T15:07:21ZengMDPI AGRemote Sensing2072-42922023-12-011618610.3390/rs16010086EGMT-CD: Edge-Guided Multimodal Transformers Change Detection from Satellite and Aerial ImagesYunfan Xiang0Xiangyu Tian1Yue Xu2Xiaokun Guan3Zhengchao Chen4Airborne Remote Sensing Center, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaAirborne Remote Sensing Center, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaState Key Laboratory of Remote Sensing Science, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaAirborne Remote Sensing Center, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaState Key Laboratory of Remote Sensing Science, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, ChinaChange detection from heterogeneous satellite and aerial images plays a progressively important role in many fields, including disaster assessment, urban construction, and land use monitoring. Currently, researchers have mainly devoted their attention to change detection using homologous image pairs and achieved many remarkable results. It is sometimes necessary to use heterogeneous images for change detection in practical scenarios due to missing images, emergency situations, and cloud and fog occlusion. However, heterogeneous change detection still faces great challenges, especially using satellite and aerial images. The main challenges in satellite and aerial image change detection are related to the resolution gap and blurred edge. Previous studies used interpolation or shallow feature alignment before traditional homologous change detection methods, which ignored the high-level feature interaction and edge information. Therefore, we propose a new heterogeneous change detection model based on multimodal transformers combined with edge guidance. In order to alleviate the resolution gap between satellite and aerial images, we design an improved spatially aligned transformer (SP-T) with a sub-pixel module to align the satellite features to the same size of the aerial ones supervised by a token loss. Moreover, we introduce an edge detection branch to guide change features using the object edge with an auxiliary edge-change loss. Finally, we conduct considerable experiments to verify the effectiveness and superiority of our proposed model (EGMT-CD) on a new satellite–aerial heterogeneous change dataset, named SACD. The experiments show that our method (EGMT-CD) outperforms many previously superior change detection methods and fully demonstrates its potential in heterogeneous change detection from satellite–aerial images.https://www.mdpi.com/2072-4292/16/1/86change detectionremote sensingheterogeneous imagesfeature alignmentedge detectiontransformer |
| spellingShingle | Yunfan Xiang Xiangyu Tian Yue Xu Xiaokun Guan Zhengchao Chen EGMT-CD: Edge-Guided Multimodal Transformers Change Detection from Satellite and Aerial Images Remote Sensing change detection remote sensing heterogeneous images feature alignment edge detection transformer |
| title | EGMT-CD: Edge-Guided Multimodal Transformers Change Detection from Satellite and Aerial Images |
| title_full | EGMT-CD: Edge-Guided Multimodal Transformers Change Detection from Satellite and Aerial Images |
| title_fullStr | EGMT-CD: Edge-Guided Multimodal Transformers Change Detection from Satellite and Aerial Images |
| title_full_unstemmed | EGMT-CD: Edge-Guided Multimodal Transformers Change Detection from Satellite and Aerial Images |
| title_short | EGMT-CD: Edge-Guided Multimodal Transformers Change Detection from Satellite and Aerial Images |
| title_sort | egmt cd edge guided multimodal transformers change detection from satellite and aerial images |
| topic | change detection remote sensing heterogeneous images feature alignment edge detection transformer |
| url | https://www.mdpi.com/2072-4292/16/1/86 |
| work_keys_str_mv | AT yunfanxiang egmtcdedgeguidedmultimodaltransformerschangedetectionfromsatelliteandaerialimages AT xiangyutian egmtcdedgeguidedmultimodaltransformerschangedetectionfromsatelliteandaerialimages AT yuexu egmtcdedgeguidedmultimodaltransformerschangedetectionfromsatelliteandaerialimages AT xiaokunguan egmtcdedgeguidedmultimodaltransformerschangedetectionfromsatelliteandaerialimages AT zhengchaochen egmtcdedgeguidedmultimodaltransformerschangedetectionfromsatelliteandaerialimages |