STDecoder-CD: How to Decode the Hierarchical Transformer in Change Detection Tasks
Change detection (CD) is in demand in satellite imagery processing. Inspired by the recent success of the combined transformer-CNN (convolutional neural network) model, TransCNN, originally designed for image recognition, in this paper, we present STDecoder-CD for change detection applications, whic...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-08-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/15/7903 |
| _version_ | 1797442710624272384 |
|---|---|
| author | Bo Zhao Xiaoyan Luo Panpan Tang Yang Liu Haoming Wan Ninglei Ouyang |
| author_facet | Bo Zhao Xiaoyan Luo Panpan Tang Yang Liu Haoming Wan Ninglei Ouyang |
| author_sort | Bo Zhao |
| collection | DOAJ |
| description | Change detection (CD) is in demand in satellite imagery processing. Inspired by the recent success of the combined transformer-CNN (convolutional neural network) model, TransCNN, originally designed for image recognition, in this paper, we present STDecoder-CD for change detection applications, which is a combination of the Siamese network (“S”), the TransCNN backbone (“T”), and three types of decoders (“Decoder”). The Type I model uses a UNet-like decoder, and the Type II decoder is defined by a combination of three modules: the difference detector, FPN (feature pyramid network), and FCN (fully convolutional network). The Type III model updates the change feature map by introducing a transformer decoder. The effectiveness and advantages of the proposed methods over the state-of-the-art alternatives were demonstrated on several CD datasets, and experimental results indicate that: (1) STDecoder-CD has excellent generalization ability and has strong robustness to pseudo-changes and noise. (2) An end-to-end CD network architecture cannot be completely free from the influence of the decoding strategy. In our case, the Type I decoder often obtained finer details than Types II and III due to its multi-scale design. (3) Using the ablation or replacing strategy to modify the three proposed decoder architectures had a limited impact on the CD performance of STDecoder-CD. To the best of our knowledge, we are the first to investigate the effect of different decoding strategies on CD tasks. |
| first_indexed | 2024-03-09T12:45:56Z |
| format | Article |
| id | doaj.art-815108444f6a4a3084af267ec56408d7 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T12:45:56Z |
| publishDate | 2022-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-815108444f6a4a3084af267ec56408d72023-11-30T22:12:16ZengMDPI AGApplied Sciences2076-34172022-08-011215790310.3390/app12157903STDecoder-CD: How to Decode the Hierarchical Transformer in Change Detection TasksBo Zhao0Xiaoyan Luo1Panpan Tang2Yang Liu3Haoming Wan4Ninglei Ouyang5Research Center of Big Data Technology, Nanhu Laboratory, Jiaxing 314000, ChinaResearch Center of Big Data Technology, Nanhu Laboratory, Jiaxing 314000, ChinaResearch Center of Big Data Technology, Nanhu Laboratory, Jiaxing 314000, ChinaResearch Center of Big Data Technology, Nanhu Laboratory, Jiaxing 314000, ChinaResearch Center of Big Data Technology, Nanhu Laboratory, Jiaxing 314000, ChinaResearch Center of Big Data Technology, Nanhu Laboratory, Jiaxing 314000, ChinaChange detection (CD) is in demand in satellite imagery processing. Inspired by the recent success of the combined transformer-CNN (convolutional neural network) model, TransCNN, originally designed for image recognition, in this paper, we present STDecoder-CD for change detection applications, which is a combination of the Siamese network (“S”), the TransCNN backbone (“T”), and three types of decoders (“Decoder”). The Type I model uses a UNet-like decoder, and the Type II decoder is defined by a combination of three modules: the difference detector, FPN (feature pyramid network), and FCN (fully convolutional network). The Type III model updates the change feature map by introducing a transformer decoder. The effectiveness and advantages of the proposed methods over the state-of-the-art alternatives were demonstrated on several CD datasets, and experimental results indicate that: (1) STDecoder-CD has excellent generalization ability and has strong robustness to pseudo-changes and noise. (2) An end-to-end CD network architecture cannot be completely free from the influence of the decoding strategy. In our case, the Type I decoder often obtained finer details than Types II and III due to its multi-scale design. (3) Using the ablation or replacing strategy to modify the three proposed decoder architectures had a limited impact on the CD performance of STDecoder-CD. To the best of our knowledge, we are the first to investigate the effect of different decoding strategies on CD tasks.https://www.mdpi.com/2076-3417/12/15/7903transformerneural networkremote sensingchange detection |
| spellingShingle | Bo Zhao Xiaoyan Luo Panpan Tang Yang Liu Haoming Wan Ninglei Ouyang STDecoder-CD: How to Decode the Hierarchical Transformer in Change Detection Tasks Applied Sciences transformer neural network remote sensing change detection |
| title | STDecoder-CD: How to Decode the Hierarchical Transformer in Change Detection Tasks |
| title_full | STDecoder-CD: How to Decode the Hierarchical Transformer in Change Detection Tasks |
| title_fullStr | STDecoder-CD: How to Decode the Hierarchical Transformer in Change Detection Tasks |
| title_full_unstemmed | STDecoder-CD: How to Decode the Hierarchical Transformer in Change Detection Tasks |
| title_short | STDecoder-CD: How to Decode the Hierarchical Transformer in Change Detection Tasks |
| title_sort | stdecoder cd how to decode the hierarchical transformer in change detection tasks |
| topic | transformer neural network remote sensing change detection |
| url | https://www.mdpi.com/2076-3417/12/15/7903 |
| work_keys_str_mv | AT bozhao stdecodercdhowtodecodethehierarchicaltransformerinchangedetectiontasks AT xiaoyanluo stdecodercdhowtodecodethehierarchicaltransformerinchangedetectiontasks AT panpantang stdecodercdhowtodecodethehierarchicaltransformerinchangedetectiontasks AT yangliu stdecodercdhowtodecodethehierarchicaltransformerinchangedetectiontasks AT haomingwan stdecodercdhowtodecodethehierarchicaltransformerinchangedetectiontasks AT ningleiouyang stdecodercdhowtodecodethehierarchicaltransformerinchangedetectiontasks |