MTSCD-Net: A network based on multi-task learning for semantic change detection of bitemporal remote sensing images
In recent years, change detection has been one of the hot research topics within the field of remote sensing. Previous studies have concentrated on binary change detection (BCD), but it doesn’t meet the current needs. Therefore, semantic change detection (SCD) is also gradually developing, which foc...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-04-01
|
| Series: | International Journal of Applied Earth Observations and Geoinformation |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1569843223001164 |
| _version_ | 1797842807157686272 |
|---|---|
| author | Fengzhi Cui Jie Jiang |
| author_facet | Fengzhi Cui Jie Jiang |
| author_sort | Fengzhi Cui |
| collection | DOAJ |
| description | In recent years, change detection has been one of the hot research topics within the field of remote sensing. Previous studies have concentrated on binary change detection (BCD), but it doesn’t meet the current needs. Therefore, semantic change detection (SCD) is also gradually developing, which focuses on determining the specific changed type while obtaining changed areas. In the paper, we propose a multi-task learning method (MTSCD-Net) for SCD task. The SCD task is decoupled into two related subtasks, semantic segmentation (SS) and BCD, then unifies them under the same framework. Multi-scale features are extracted using the Siamese semantic-aware encoder based on Swin Transformer, and the aggregation module is designed to combine features. Then, the change information extraction module is designed to enhance the capacity to express features by fully integrating the two-level difference features that are generated from fused features. Moreover, in the decoder stage, the spatial attention weight map is obtained using the features of the BCD subtask, which provides location prior information for the features of the SS subtask. It helps fully explore the correlation between the two subtasks. The two loss functions of subtasks are weighted to train MTSCD-Net. The comparative experiments results on two typical SCD datasets confirm the advantage of MTSCD-Net for SCD task. For the SeK index, MTSCD-Net achieves 3.96% and 20.57% on HRSCD and SECOND datasets, respectively. This outperforms other comparative methods such as Bi-SRNet (which achieves 4.86% and 1.47% higher on two datasets, respectively). The same is true for the Score metric. Moreover, the ablation experiment results confirm the effectiveness of key modules. |
| first_indexed | 2024-04-09T16:55:20Z |
| format | Article |
| id | doaj.art-e893d589ac8e46d8871113bb0875b4d5 |
| institution | Directory Open Access Journal |
| issn | 1569-8432 |
| language | English |
| last_indexed | 2024-04-09T16:55:20Z |
| publishDate | 2023-04-01 |
| publisher | Elsevier |
| record_format | Article |
| series | International Journal of Applied Earth Observations and Geoinformation |
| spelling | doaj.art-e893d589ac8e46d8871113bb0875b4d52023-04-21T06:43:23ZengElsevierInternational Journal of Applied Earth Observations and Geoinformation1569-84322023-04-01118103294MTSCD-Net: A network based on multi-task learning for semantic change detection of bitemporal remote sensing imagesFengzhi Cui0Jie Jiang1School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China; Key Laboratory of Precision Opto-Mechatronics Technology, Ministry of Education, Beihang University, Beijing 100191, ChinaSchool of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China; Key Laboratory of Precision Opto-Mechatronics Technology, Ministry of Education, Beihang University, Beijing 100191, China; Corresponding author at: School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China.In recent years, change detection has been one of the hot research topics within the field of remote sensing. Previous studies have concentrated on binary change detection (BCD), but it doesn’t meet the current needs. Therefore, semantic change detection (SCD) is also gradually developing, which focuses on determining the specific changed type while obtaining changed areas. In the paper, we propose a multi-task learning method (MTSCD-Net) for SCD task. The SCD task is decoupled into two related subtasks, semantic segmentation (SS) and BCD, then unifies them under the same framework. Multi-scale features are extracted using the Siamese semantic-aware encoder based on Swin Transformer, and the aggregation module is designed to combine features. Then, the change information extraction module is designed to enhance the capacity to express features by fully integrating the two-level difference features that are generated from fused features. Moreover, in the decoder stage, the spatial attention weight map is obtained using the features of the BCD subtask, which provides location prior information for the features of the SS subtask. It helps fully explore the correlation between the two subtasks. The two loss functions of subtasks are weighted to train MTSCD-Net. The comparative experiments results on two typical SCD datasets confirm the advantage of MTSCD-Net for SCD task. For the SeK index, MTSCD-Net achieves 3.96% and 20.57% on HRSCD and SECOND datasets, respectively. This outperforms other comparative methods such as Bi-SRNet (which achieves 4.86% and 1.47% higher on two datasets, respectively). The same is true for the Score metric. Moreover, the ablation experiment results confirm the effectiveness of key modules.http://www.sciencedirect.com/science/article/pii/S1569843223001164Remote sensingMulti-task learningSiamese networkDeep learningSemantic change detection |
| spellingShingle | Fengzhi Cui Jie Jiang MTSCD-Net: A network based on multi-task learning for semantic change detection of bitemporal remote sensing images International Journal of Applied Earth Observations and Geoinformation Remote sensing Multi-task learning Siamese network Deep learning Semantic change detection |
| title | MTSCD-Net: A network based on multi-task learning for semantic change detection of bitemporal remote sensing images |
| title_full | MTSCD-Net: A network based on multi-task learning for semantic change detection of bitemporal remote sensing images |
| title_fullStr | MTSCD-Net: A network based on multi-task learning for semantic change detection of bitemporal remote sensing images |
| title_full_unstemmed | MTSCD-Net: A network based on multi-task learning for semantic change detection of bitemporal remote sensing images |
| title_short | MTSCD-Net: A network based on multi-task learning for semantic change detection of bitemporal remote sensing images |
| title_sort | mtscd net a network based on multi task learning for semantic change detection of bitemporal remote sensing images |
| topic | Remote sensing Multi-task learning Siamese network Deep learning Semantic change detection |
| url | http://www.sciencedirect.com/science/article/pii/S1569843223001164 |
| work_keys_str_mv | AT fengzhicui mtscdnetanetworkbasedonmultitasklearningforsemanticchangedetectionofbitemporalremotesensingimages AT jiejiang mtscdnetanetworkbasedonmultitasklearningforsemanticchangedetectionofbitemporalremotesensingimages |