A transformer-based Siamese network and an open optical dataset for semantic change detection of remote sensing images
Recent change detection (CD) methods focus on the extraction of deep change semantic features. However, existing methods overlook the fine-grained features and have the poor ability to capture long-range space–time information, which leads to the micro changes missing and the edges of change types s...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2022-12-01
|
| Series: | International Journal of Digital Earth |
| Subjects: | |
| Online Access: | http://dx.doi.org/10.1080/17538947.2022.2111470 |
| _version_ | 1797678517418196992 |
|---|---|
| author | Panli Yuan Qingzhan Zhao Xingbiao Zhao Xuewen Wang Xuefeng Long Yuchen Zheng |
| author_facet | Panli Yuan Qingzhan Zhao Xingbiao Zhao Xuewen Wang Xuefeng Long Yuchen Zheng |
| author_sort | Panli Yuan |
| collection | DOAJ |
| description | Recent change detection (CD) methods focus on the extraction of deep change semantic features. However, existing methods overlook the fine-grained features and have the poor ability to capture long-range space–time information, which leads to the micro changes missing and the edges of change types smoothing. In this paper, a potential transformer-based semantic change detection (SCD) model, Pyramid-SCDFormer is proposed, which precisely recognizes the small changes and fine edges details of the changes. The SCD model selectively merges different semantic tokens in multi-head self-attention block to obtain multiscale features, which is crucial for extraction information of remote sensing images (RSIs) with multiple changes from different scales. Moreover, we create a well-annotated SCD dataset, Landsat-SCD with unprecedented time series and change types in complex scenarios. Comparing with three Convolutional Neural Network-based, one attention-based, and two transformer-based networks, experimental results demonstrate that the Pyramid-SCDFormer stably outperforms the existing state-of-the-art CD models and obtains an improvement in MIoU/F1 of 1.11/0.76%, 0.57/0.50%, and 8.75/8.59% on the LEVIR-CD, WHU_CD, and Landsat-SCD dataset respectively. For change classes proportion less than 1%, the proposed model improves the MIoU by 7.17–19.53% on Landsat-SCD dataset. The recognition performance for small-scale and fine edges of change types has greatly improved. |
| first_indexed | 2024-03-11T23:00:57Z |
| format | Article |
| id | doaj.art-d8a0c4838078459995aa8ef873518f42 |
| institution | Directory Open Access Journal |
| issn | 1753-8947 1753-8955 |
| language | English |
| last_indexed | 2024-03-11T23:00:57Z |
| publishDate | 2022-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | International Journal of Digital Earth |
| spelling | doaj.art-d8a0c4838078459995aa8ef873518f422023-09-21T14:57:11ZengTaylor & Francis GroupInternational Journal of Digital Earth1753-89471753-89552022-12-011511506152510.1080/17538947.2022.21114702111470A transformer-based Siamese network and an open optical dataset for semantic change detection of remote sensing imagesPanli Yuan0Qingzhan Zhao1Xingbiao Zhao2Xuewen Wang3Xuefeng Long4Yuchen Zheng5Shihezi UniversityShihezi UniversityShihezi UniversityInstitute of Geophysics and Geomatics, China University of GeosciencesShihezi UniversityShihezi UniversityRecent change detection (CD) methods focus on the extraction of deep change semantic features. However, existing methods overlook the fine-grained features and have the poor ability to capture long-range space–time information, which leads to the micro changes missing and the edges of change types smoothing. In this paper, a potential transformer-based semantic change detection (SCD) model, Pyramid-SCDFormer is proposed, which precisely recognizes the small changes and fine edges details of the changes. The SCD model selectively merges different semantic tokens in multi-head self-attention block to obtain multiscale features, which is crucial for extraction information of remote sensing images (RSIs) with multiple changes from different scales. Moreover, we create a well-annotated SCD dataset, Landsat-SCD with unprecedented time series and change types in complex scenarios. Comparing with three Convolutional Neural Network-based, one attention-based, and two transformer-based networks, experimental results demonstrate that the Pyramid-SCDFormer stably outperforms the existing state-of-the-art CD models and obtains an improvement in MIoU/F1 of 1.11/0.76%, 0.57/0.50%, and 8.75/8.59% on the LEVIR-CD, WHU_CD, and Landsat-SCD dataset respectively. For change classes proportion less than 1%, the proposed model improves the MIoU by 7.17–19.53% on Landsat-SCD dataset. The recognition performance for small-scale and fine edges of change types has greatly improved.http://dx.doi.org/10.1080/17538947.2022.2111470semantic change detection (scd)change detection datasettransformer siamese networkself-attention mechanismbitemporal remote sensing |
| spellingShingle | Panli Yuan Qingzhan Zhao Xingbiao Zhao Xuewen Wang Xuefeng Long Yuchen Zheng A transformer-based Siamese network and an open optical dataset for semantic change detection of remote sensing images International Journal of Digital Earth semantic change detection (scd) change detection dataset transformer siamese network self-attention mechanism bitemporal remote sensing |
| title | A transformer-based Siamese network and an open optical dataset for semantic change detection of remote sensing images |
| title_full | A transformer-based Siamese network and an open optical dataset for semantic change detection of remote sensing images |
| title_fullStr | A transformer-based Siamese network and an open optical dataset for semantic change detection of remote sensing images |
| title_full_unstemmed | A transformer-based Siamese network and an open optical dataset for semantic change detection of remote sensing images |
| title_short | A transformer-based Siamese network and an open optical dataset for semantic change detection of remote sensing images |
| title_sort | transformer based siamese network and an open optical dataset for semantic change detection of remote sensing images |
| topic | semantic change detection (scd) change detection dataset transformer siamese network self-attention mechanism bitemporal remote sensing |
| url | http://dx.doi.org/10.1080/17538947.2022.2111470 |
| work_keys_str_mv | AT panliyuan atransformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT qingzhanzhao atransformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT xingbiaozhao atransformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT xuewenwang atransformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT xuefenglong atransformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT yuchenzheng atransformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT panliyuan transformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT qingzhanzhao transformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT xingbiaozhao transformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT xuewenwang transformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT xuefenglong transformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages AT yuchenzheng transformerbasedsiamesenetworkandanopenopticaldatasetforsemanticchangedetectionofremotesensingimages |