Multi-Scale Residual Spectral–Spatial Attention Combined with Improved Transformer for Hyperspectral Image Classification
Aiming to solve the problems of different spectral bands and spatial pixels contributing differently to hyperspectral image (HSI) classification, and sparse connectivity restricting the convolutional neural network to a globally dependent capture, we propose a HSI classification model combined with...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/13/6/1061 |
| _version_ | 1797241338952941568 |
|---|---|
| author | Aili Wang Kang Zhang Haibin Wu Yuji Iwahori Haisong Chen |
| author_facet | Aili Wang Kang Zhang Haibin Wu Yuji Iwahori Haisong Chen |
| author_sort | Aili Wang |
| collection | DOAJ |
| description | Aiming to solve the problems of different spectral bands and spatial pixels contributing differently to hyperspectral image (HSI) classification, and sparse connectivity restricting the convolutional neural network to a globally dependent capture, we propose a HSI classification model combined with multi-scale residual spectral–spatial attention and an improved transformer in this paper. First, in order to efficiently highlight discriminative spectral–spatial information, we propose a multi-scale residual spectral–spatial feature extraction module that preserves the multi-scale information in a two-layer cascade structure, and the spectral–spatial features are refined by residual spectral–spatial attention for the feature-learning stage. In addition, to further capture the sequential spectral relationships, we combine the advantages of Cross-Attention and Re-Attention to alleviate computational burden and attention collapse issues, and propose the Cross-Re-Attention mechanism to achieve an improved transformer, which can efficiently alleviate the heavy memory footprint and huge computational burden of the model. The experimental results show that the overall accuracy of the proposed model in this paper can reach 98.71%, 99.33%, and 99.72% for Indiana Pines, Kennedy Space Center, and XuZhou datasets, respectively. The proposed method was verified to have high accuracy and effectiveness compared to the state-of-the-art models, which shows that the concept of the hybrid architecture opens a new window for HSI classification. |
| first_indexed | 2024-04-24T18:21:45Z |
| format | Article |
| id | doaj.art-d0c8bbfe29124859a5f695c7078b20fc |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-04-24T18:21:45Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-d0c8bbfe29124859a5f695c7078b20fc2024-03-27T13:34:52ZengMDPI AGElectronics2079-92922024-03-01136106110.3390/electronics13061061Multi-Scale Residual Spectral–Spatial Attention Combined with Improved Transformer for Hyperspectral Image ClassificationAili Wang0Kang Zhang1Haibin Wu2Yuji Iwahori3Haisong Chen4Heilongjiang Province Key Laboratory of Laser Spectroscopy Technology and Application, Harbin University of Science and Technology, Harbin 150080, ChinaHeilongjiang Province Key Laboratory of Laser Spectroscopy Technology and Application, Harbin University of Science and Technology, Harbin 150080, ChinaHeilongjiang Province Key Laboratory of Laser Spectroscopy Technology and Application, Harbin University of Science and Technology, Harbin 150080, ChinaComputer Science, Chubu University, Kasugai 487-8501, JapanSchool of Undergraduate Education, Shenzhen Polytechnic University, Shenzhen 518115, ChinaAiming to solve the problems of different spectral bands and spatial pixels contributing differently to hyperspectral image (HSI) classification, and sparse connectivity restricting the convolutional neural network to a globally dependent capture, we propose a HSI classification model combined with multi-scale residual spectral–spatial attention and an improved transformer in this paper. First, in order to efficiently highlight discriminative spectral–spatial information, we propose a multi-scale residual spectral–spatial feature extraction module that preserves the multi-scale information in a two-layer cascade structure, and the spectral–spatial features are refined by residual spectral–spatial attention for the feature-learning stage. In addition, to further capture the sequential spectral relationships, we combine the advantages of Cross-Attention and Re-Attention to alleviate computational burden and attention collapse issues, and propose the Cross-Re-Attention mechanism to achieve an improved transformer, which can efficiently alleviate the heavy memory footprint and huge computational burden of the model. The experimental results show that the overall accuracy of the proposed model in this paper can reach 98.71%, 99.33%, and 99.72% for Indiana Pines, Kennedy Space Center, and XuZhou datasets, respectively. The proposed method was verified to have high accuracy and effectiveness compared to the state-of-the-art models, which shows that the concept of the hybrid architecture opens a new window for HSI classification.https://www.mdpi.com/2079-9292/13/6/1061hyperspectral image classificationmulti-scale feature extractionresidual spectral–spatial attentiontransformer |
| spellingShingle | Aili Wang Kang Zhang Haibin Wu Yuji Iwahori Haisong Chen Multi-Scale Residual Spectral–Spatial Attention Combined with Improved Transformer for Hyperspectral Image Classification Electronics hyperspectral image classification multi-scale feature extraction residual spectral–spatial attention transformer |
| title | Multi-Scale Residual Spectral–Spatial Attention Combined with Improved Transformer for Hyperspectral Image Classification |
| title_full | Multi-Scale Residual Spectral–Spatial Attention Combined with Improved Transformer for Hyperspectral Image Classification |
| title_fullStr | Multi-Scale Residual Spectral–Spatial Attention Combined with Improved Transformer for Hyperspectral Image Classification |
| title_full_unstemmed | Multi-Scale Residual Spectral–Spatial Attention Combined with Improved Transformer for Hyperspectral Image Classification |
| title_short | Multi-Scale Residual Spectral–Spatial Attention Combined with Improved Transformer for Hyperspectral Image Classification |
| title_sort | multi scale residual spectral spatial attention combined with improved transformer for hyperspectral image classification |
| topic | hyperspectral image classification multi-scale feature extraction residual spectral–spatial attention transformer |
| url | https://www.mdpi.com/2079-9292/13/6/1061 |
| work_keys_str_mv | AT ailiwang multiscaleresidualspectralspatialattentioncombinedwithimprovedtransformerforhyperspectralimageclassification AT kangzhang multiscaleresidualspectralspatialattentioncombinedwithimprovedtransformerforhyperspectralimageclassification AT haibinwu multiscaleresidualspectralspatialattentioncombinedwithimprovedtransformerforhyperspectralimageclassification AT yujiiwahori multiscaleresidualspectralspatialattentioncombinedwithimprovedtransformerforhyperspectralimageclassification AT haisongchen multiscaleresidualspectralspatialattentioncombinedwithimprovedtransformerforhyperspectralimageclassification |