Shallow-Guided Transformer for Semantic Segmentation of Hyperspectral Remote Sensing Imagery
Convolutional neural networks (CNNs) have achieved great progress in the classification of surface objects with hyperspectral data, but due to the limitations of convolutional operations, CNNs cannot effectively interact with contextual information. Transformer succeeds in solving this problem, and...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/13/3366 |
| _version_ | 1827734563109994496 |
|---|---|
| author | Yuhan Chen Pengyuan Liu Jiechen Zhao Kaijian Huang Qingyun Yan |
| author_facet | Yuhan Chen Pengyuan Liu Jiechen Zhao Kaijian Huang Qingyun Yan |
| author_sort | Yuhan Chen |
| collection | DOAJ |
| description | Convolutional neural networks (CNNs) have achieved great progress in the classification of surface objects with hyperspectral data, but due to the limitations of convolutional operations, CNNs cannot effectively interact with contextual information. Transformer succeeds in solving this problem, and thus has been widely used to classify hyperspectral surface objects in recent years. However, the huge computational load of Transformer poses a challenge in hyperspectral semantic segmentation tasks. In addition, the use of single Transformer discards the local correlation, making it ineffective for remote sensing tasks with small datasets. Therefore, we propose a new Transformer layered architecture that combines Transformer with CNN, adopts a feature dimensionality reduction module and a Transformer-style CNN module to extract shallow features and construct texture constraints, and employs the original Transformer Encoder to extract deep features. Furthermore, we also designed a simple Decoder to process shallow spatial detail information and deep semantic features separately. Experimental results based on three publicly available hyperspectral datasets show that our proposed method has significant advantages compared with other traditional CNN, Transformer-type models. |
| first_indexed | 2024-03-11T01:30:20Z |
| format | Article |
| id | doaj.art-6e432fc100a14871bbd32489389649e3 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-11T01:30:20Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-6e432fc100a14871bbd32489389649e32023-11-18T17:25:13ZengMDPI AGRemote Sensing2072-42922023-06-011513336610.3390/rs15133366Shallow-Guided Transformer for Semantic Segmentation of Hyperspectral Remote Sensing ImageryYuhan Chen0Pengyuan Liu1Jiechen Zhao2Kaijian Huang3Qingyun Yan4School of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, ChinaSchool of Geographical Sciences, Nanjing University of Information Science and Technology, Nanjing 210044, ChinaQingdao Innovation and Development Base (Centre), Harbin Engineering University, Qingdao 266000, ChinaSchool of Electronic Information and Electrical Engineering, Huizhou University, Huizhou 516007, ChinaSchool of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science and Technology, Nanjing 210044, ChinaConvolutional neural networks (CNNs) have achieved great progress in the classification of surface objects with hyperspectral data, but due to the limitations of convolutional operations, CNNs cannot effectively interact with contextual information. Transformer succeeds in solving this problem, and thus has been widely used to classify hyperspectral surface objects in recent years. However, the huge computational load of Transformer poses a challenge in hyperspectral semantic segmentation tasks. In addition, the use of single Transformer discards the local correlation, making it ineffective for remote sensing tasks with small datasets. Therefore, we propose a new Transformer layered architecture that combines Transformer with CNN, adopts a feature dimensionality reduction module and a Transformer-style CNN module to extract shallow features and construct texture constraints, and employs the original Transformer Encoder to extract deep features. Furthermore, we also designed a simple Decoder to process shallow spatial detail information and deep semantic features separately. Experimental results based on three publicly available hyperspectral datasets show that our proposed method has significant advantages compared with other traditional CNN, Transformer-type models.https://www.mdpi.com/2072-4292/15/13/3366vision transformerconvolutional neural networks (CNNs)feature representationshyperspectral images (HSIs)semantic segmentation |
| spellingShingle | Yuhan Chen Pengyuan Liu Jiechen Zhao Kaijian Huang Qingyun Yan Shallow-Guided Transformer for Semantic Segmentation of Hyperspectral Remote Sensing Imagery Remote Sensing vision transformer convolutional neural networks (CNNs) feature representations hyperspectral images (HSIs) semantic segmentation |
| title | Shallow-Guided Transformer for Semantic Segmentation of Hyperspectral Remote Sensing Imagery |
| title_full | Shallow-Guided Transformer for Semantic Segmentation of Hyperspectral Remote Sensing Imagery |
| title_fullStr | Shallow-Guided Transformer for Semantic Segmentation of Hyperspectral Remote Sensing Imagery |
| title_full_unstemmed | Shallow-Guided Transformer for Semantic Segmentation of Hyperspectral Remote Sensing Imagery |
| title_short | Shallow-Guided Transformer for Semantic Segmentation of Hyperspectral Remote Sensing Imagery |
| title_sort | shallow guided transformer for semantic segmentation of hyperspectral remote sensing imagery |
| topic | vision transformer convolutional neural networks (CNNs) feature representations hyperspectral images (HSIs) semantic segmentation |
| url | https://www.mdpi.com/2072-4292/15/13/3366 |
| work_keys_str_mv | AT yuhanchen shallowguidedtransformerforsemanticsegmentationofhyperspectralremotesensingimagery AT pengyuanliu shallowguidedtransformerforsemanticsegmentationofhyperspectralremotesensingimagery AT jiechenzhao shallowguidedtransformerforsemanticsegmentationofhyperspectralremotesensingimagery AT kaijianhuang shallowguidedtransformerforsemanticsegmentationofhyperspectralremotesensingimagery AT qingyunyan shallowguidedtransformerforsemanticsegmentationofhyperspectralremotesensingimagery |