Spectral Token Guidance Transformer for Multisource Images Change Detection
With the development of Earth observation technology, more multisource remote sensing images are obtained from various satellite sensors and significantly enrich the data source of change detection (CD). However, the utilization of multisource bitemporal images frequently introduces challenges durin...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10058182/ |
| _version_ | 1797866197998370816 |
|---|---|
| author | Bangyong Sun Qinsen Liu Nianzeng Yuan Jiahai Tan Xiaomei Gao Tao Yu |
| author_facet | Bangyong Sun Qinsen Liu Nianzeng Yuan Jiahai Tan Xiaomei Gao Tao Yu |
| author_sort | Bangyong Sun |
| collection | DOAJ |
| description | With the development of Earth observation technology, more multisource remote sensing images are obtained from various satellite sensors and significantly enrich the data source of change detection (CD). However, the utilization of multisource bitemporal images frequently introduces challenges during featuring or representing the various physical mechanisms of the observed landscapes and makes it more difficult to develop a general model for homogeneous and heterogeneous CD adaptively. In this article, we propose an adaptive spatial-spectral transformer CD network based on spectral token guidance, named STCD-Former. Specifically, a spectral transformer with dual-branch first encodes the diverse spectral sequence in spectral-wise to generate a corresponding spectral token. And then, the spectral token is used as guidance to interact with the patch token to learn the change rules. More significantly, to optimize the learning of difference information, we design a difference amplification module to highlight discriminative features by adaptively integrating the difference information into the feature embedding. Finally, the binary CD result is obtained by multilayer perceptron. The experimental results on three homogeneous datasets and one heterogeneous dataset have demonstrated that the proposed STCD-Former outperforms the other state-of-the-art methods qualitatively and visually. |
| first_indexed | 2024-04-09T23:21:13Z |
| format | Article |
| id | doaj.art-e0fdbfb3daa74d7abe0c207837ab0d59 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-04-09T23:21:13Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-e0fdbfb3daa74d7abe0c207837ab0d592023-03-21T23:00:14ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352023-01-01162559257210.1109/JSTARS.2023.325196210058182Spectral Token Guidance Transformer for Multisource Images Change DetectionBangyong Sun0https://orcid.org/0000-0002-0265-1785Qinsen Liu1Nianzeng Yuan2Jiahai Tan3Xiaomei Gao4Tao Yu5https://orcid.org/0009-0003-5701-9642School of Printing, Packaging, and Digital Media, Xi'an University of Technology, Xi'an, ChinaSchool of Printing, Packaging, and Digital Media, Xi'an University of Technology, Xi'an, ChinaSchool of Printing, Packaging, and Digital Media, Xi'an University of Technology, Xi'an, ChinaSchool of Optoelectronic Engineering, Xi'an Technological University, Xi'an, ChinaMapping and Printing of China National Administration of Coal Geology, Xi'an, ChinaKey Laboratory of Spectral Imaging Technology CAS, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, ChinaWith the development of Earth observation technology, more multisource remote sensing images are obtained from various satellite sensors and significantly enrich the data source of change detection (CD). However, the utilization of multisource bitemporal images frequently introduces challenges during featuring or representing the various physical mechanisms of the observed landscapes and makes it more difficult to develop a general model for homogeneous and heterogeneous CD adaptively. In this article, we propose an adaptive spatial-spectral transformer CD network based on spectral token guidance, named STCD-Former. Specifically, a spectral transformer with dual-branch first encodes the diverse spectral sequence in spectral-wise to generate a corresponding spectral token. And then, the spectral token is used as guidance to interact with the patch token to learn the change rules. More significantly, to optimize the learning of difference information, we design a difference amplification module to highlight discriminative features by adaptively integrating the difference information into the feature embedding. Finally, the binary CD result is obtained by multilayer perceptron. The experimental results on three homogeneous datasets and one heterogeneous dataset have demonstrated that the proposed STCD-Former outperforms the other state-of-the-art methods qualitatively and visually.https://ieeexplore.ieee.org/document/10058182/Change detection (CD)heterogeneous imageshyperspectral imagesmultispectral imagestransformer |
| spellingShingle | Bangyong Sun Qinsen Liu Nianzeng Yuan Jiahai Tan Xiaomei Gao Tao Yu Spectral Token Guidance Transformer for Multisource Images Change Detection IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Change detection (CD) heterogeneous images hyperspectral images multispectral images transformer |
| title | Spectral Token Guidance Transformer for Multisource Images Change Detection |
| title_full | Spectral Token Guidance Transformer for Multisource Images Change Detection |
| title_fullStr | Spectral Token Guidance Transformer for Multisource Images Change Detection |
| title_full_unstemmed | Spectral Token Guidance Transformer for Multisource Images Change Detection |
| title_short | Spectral Token Guidance Transformer for Multisource Images Change Detection |
| title_sort | spectral token guidance transformer for multisource images change detection |
| topic | Change detection (CD) heterogeneous images hyperspectral images multispectral images transformer |
| url | https://ieeexplore.ieee.org/document/10058182/ |
| work_keys_str_mv | AT bangyongsun spectraltokenguidancetransformerformultisourceimageschangedetection AT qinsenliu spectraltokenguidancetransformerformultisourceimageschangedetection AT nianzengyuan spectraltokenguidancetransformerformultisourceimageschangedetection AT jiahaitan spectraltokenguidancetransformerformultisourceimageschangedetection AT xiaomeigao spectraltokenguidancetransformerformultisourceimageschangedetection AT taoyu spectraltokenguidancetransformerformultisourceimageschangedetection |