Two-Stream Swin Transformer with Differentiable Sobel Operator for Remote Sensing Image Classification
Remote sensing (RS) image classification has attracted much attention recently and is widely used in various fields. Different to natural images, the RS image scenes consist of complex backgrounds and various stochastically arranged objects, thus making it difficult for networks to focus on the targ...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
MDPI AG
2022-03-01
|
Series: | Remote Sensing |
Subjects: | |
Online Access: | https://www.mdpi.com/2072-4292/14/6/1507 |
_version_ | 1797442545299488768 |
---|---|
author | Siyuan Hao Bin Wu Kun Zhao Yuanxin Ye Wei Wang |
author_facet | Siyuan Hao Bin Wu Kun Zhao Yuanxin Ye Wei Wang |
author_sort | Siyuan Hao |
collection | DOAJ |
description | Remote sensing (RS) image classification has attracted much attention recently and is widely used in various fields. Different to natural images, the RS image scenes consist of complex backgrounds and various stochastically arranged objects, thus making it difficult for networks to focus on the target objects in the scene. However, conventional classification methods do not have any special treatment for remote sensing images. In this paper, we propose a two-stream swin transformer network (TSTNet) to address these issues. TSTNet consists of two streams (i.e., original stream and edge stream) which use both the deep features of the original images and the ones from the edges to make predictions. The swin transformer is used as the backbone of each stream given its good performance. In addition, a differentiable edge Sobel operator module (DESOM) is included in the edge stream which can learn the parameters of Sobel operator adaptively and provide more robust edge information that can suppress background noise. Experimental results on three publicly available remote sensing datasets show that our TSTNet achieves superior performance over the state-of-the-art (SOTA) methods. |
first_indexed | 2024-03-09T12:44:28Z |
format | Article |
id | doaj.art-03de899bb81d4802a9381dc2413f18be |
institution | Directory Open Access Journal |
issn | 2072-4292 |
language | English |
last_indexed | 2024-03-09T12:44:28Z |
publishDate | 2022-03-01 |
publisher | MDPI AG |
record_format | Article |
series | Remote Sensing |
spelling | doaj.art-03de899bb81d4802a9381dc2413f18be2023-11-30T22:14:09ZengMDPI AGRemote Sensing2072-42922022-03-01146150710.3390/rs14061507Two-Stream Swin Transformer with Differentiable Sobel Operator for Remote Sensing Image ClassificationSiyuan Hao0Bin Wu1Kun Zhao2Yuanxin Ye3Wei Wang4The College of Communication and Electronic Engineering, Qingdao University of Technology, Qingdao 266520, ChinaThe College of Communication and Electronic Engineering, Qingdao University of Technology, Qingdao 266520, ChinaThe College of Communication and Electronic Engineering, Qingdao University of Technology, Qingdao 266520, ChinaThe Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, ChinaThe Department of Information Engineering and Computer Science, University of Trento, Via Sommarive, 38123 Trento, ItalyRemote sensing (RS) image classification has attracted much attention recently and is widely used in various fields. Different to natural images, the RS image scenes consist of complex backgrounds and various stochastically arranged objects, thus making it difficult for networks to focus on the target objects in the scene. However, conventional classification methods do not have any special treatment for remote sensing images. In this paper, we propose a two-stream swin transformer network (TSTNet) to address these issues. TSTNet consists of two streams (i.e., original stream and edge stream) which use both the deep features of the original images and the ones from the edges to make predictions. The swin transformer is used as the backbone of each stream given its good performance. In addition, a differentiable edge Sobel operator module (DESOM) is included in the edge stream which can learn the parameters of Sobel operator adaptively and provide more robust edge information that can suppress background noise. Experimental results on three publicly available remote sensing datasets show that our TSTNet achieves superior performance over the state-of-the-art (SOTA) methods.https://www.mdpi.com/2072-4292/14/6/1507remote sensingscene classificationdeep learningswin transformerfeature fusionedge detection |
spellingShingle | Siyuan Hao Bin Wu Kun Zhao Yuanxin Ye Wei Wang Two-Stream Swin Transformer with Differentiable Sobel Operator for Remote Sensing Image Classification Remote Sensing remote sensing scene classification deep learning swin transformer feature fusion edge detection |
title | Two-Stream Swin Transformer with Differentiable Sobel Operator for Remote Sensing Image Classification |
title_full | Two-Stream Swin Transformer with Differentiable Sobel Operator for Remote Sensing Image Classification |
title_fullStr | Two-Stream Swin Transformer with Differentiable Sobel Operator for Remote Sensing Image Classification |
title_full_unstemmed | Two-Stream Swin Transformer with Differentiable Sobel Operator for Remote Sensing Image Classification |
title_short | Two-Stream Swin Transformer with Differentiable Sobel Operator for Remote Sensing Image Classification |
title_sort | two stream swin transformer with differentiable sobel operator for remote sensing image classification |
topic | remote sensing scene classification deep learning swin transformer feature fusion edge detection |
url | https://www.mdpi.com/2072-4292/14/6/1507 |
work_keys_str_mv | AT siyuanhao twostreamswintransformerwithdifferentiablesobeloperatorforremotesensingimageclassification AT binwu twostreamswintransformerwithdifferentiablesobeloperatorforremotesensingimageclassification AT kunzhao twostreamswintransformerwithdifferentiablesobeloperatorforremotesensingimageclassification AT yuanxinye twostreamswintransformerwithdifferentiablesobeloperatorforremotesensingimageclassification AT weiwang twostreamswintransformerwithdifferentiablesobeloperatorforremotesensingimageclassification |