Spatial-spectral hierarchical vision permutator for hyperspectral image classification
ABSTRACTIn recent years, the convolutional neural network (CNN) has been widely applied to hyperspectral image classification because of its powerful feature capture ability. Nevertheless, the performance of most convolutional operations is limited by the fixed shape and size of the convolutional ke...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2023-12-01
|
| Series: | European Journal of Remote Sensing |
| Subjects: | |
| Online Access: | https://www.tandfonline.com/doi/10.1080/22797254.2022.2153747 |
| _version_ | 1797946809700581376 |
|---|---|
| author | Lifan Ji Yihao Shao Jianjun Liu Liang Xiao |
| author_facet | Lifan Ji Yihao Shao Jianjun Liu Liang Xiao |
| author_sort | Lifan Ji |
| collection | DOAJ |
| description | ABSTRACTIn recent years, the convolutional neural network (CNN) has been widely applied to hyperspectral image classification because of its powerful feature capture ability. Nevertheless, the performance of most convolutional operations is limited by the fixed shape and size of the convolutional kernel, which causes CNN cannot fully extract global features. To address this issue, this article proposes a novel classification architecture named spatial-spectral hierarchical Vision Permutator (S2HViP). It contains a spectral module and a spatial module. In the spectral module, we divide the data into groups along the spectral dimension and treat each pixel within the group as a spectral token. Spectral long-range dependencies are obtained by fusing intra- and inter-group spectral correlations captured by multi-layer perceptrons (MLPs). In the spatial module, we first model spatial information via morphological methods and divide the resulting spatial feature maps into spatial tokens of uniform size. Then, the global spatial information is extracted through MLPs. Finally, the extracted spectral and spatial features are combined for classification. Particularly, the proposed MLP structure is an improved Vision Permutator, which presents a hierarchical fusion strategy aiming at generating discriminative features. Experimental results show that S2HViP can provide competitive performance compared to several state-of-the-art methods. |
| first_indexed | 2024-04-10T21:16:54Z |
| format | Article |
| id | doaj.art-e93e53b365dd4b14a808825e80d6f658 |
| institution | Directory Open Access Journal |
| issn | 2279-7254 |
| language | English |
| last_indexed | 2024-04-10T21:16:54Z |
| publishDate | 2023-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | European Journal of Remote Sensing |
| spelling | doaj.art-e93e53b365dd4b14a808825e80d6f6582023-01-20T11:18:21ZengTaylor & Francis GroupEuropean Journal of Remote Sensing2279-72542023-12-0156110.1080/22797254.2022.2153747Spatial-spectral hierarchical vision permutator for hyperspectral image classificationLifan Ji0Yihao Shao1Jianjun Liu2Liang Xiao3School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu, ChinaSchool of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu, ChinaSchool of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu, ChinaSchool of Computer Science, Nanjing University of Science and Technology, Nanjing, Jiangsu, ChinaABSTRACTIn recent years, the convolutional neural network (CNN) has been widely applied to hyperspectral image classification because of its powerful feature capture ability. Nevertheless, the performance of most convolutional operations is limited by the fixed shape and size of the convolutional kernel, which causes CNN cannot fully extract global features. To address this issue, this article proposes a novel classification architecture named spatial-spectral hierarchical Vision Permutator (S2HViP). It contains a spectral module and a spatial module. In the spectral module, we divide the data into groups along the spectral dimension and treat each pixel within the group as a spectral token. Spectral long-range dependencies are obtained by fusing intra- and inter-group spectral correlations captured by multi-layer perceptrons (MLPs). In the spatial module, we first model spatial information via morphological methods and divide the resulting spatial feature maps into spatial tokens of uniform size. Then, the global spatial information is extracted through MLPs. Finally, the extracted spectral and spatial features are combined for classification. Particularly, the proposed MLP structure is an improved Vision Permutator, which presents a hierarchical fusion strategy aiming at generating discriminative features. Experimental results show that S2HViP can provide competitive performance compared to several state-of-the-art methods.https://www.tandfonline.com/doi/10.1080/22797254.2022.2153747Hyperspectral image classificationvision permutatorfeature fusionlong-range dependencies |
| spellingShingle | Lifan Ji Yihao Shao Jianjun Liu Liang Xiao Spatial-spectral hierarchical vision permutator for hyperspectral image classification European Journal of Remote Sensing Hyperspectral image classification vision permutator feature fusion long-range dependencies |
| title | Spatial-spectral hierarchical vision permutator for hyperspectral image classification |
| title_full | Spatial-spectral hierarchical vision permutator for hyperspectral image classification |
| title_fullStr | Spatial-spectral hierarchical vision permutator for hyperspectral image classification |
| title_full_unstemmed | Spatial-spectral hierarchical vision permutator for hyperspectral image classification |
| title_short | Spatial-spectral hierarchical vision permutator for hyperspectral image classification |
| title_sort | spatial spectral hierarchical vision permutator for hyperspectral image classification |
| topic | Hyperspectral image classification vision permutator feature fusion long-range dependencies |
| url | https://www.tandfonline.com/doi/10.1080/22797254.2022.2153747 |
| work_keys_str_mv | AT lifanji spatialspectralhierarchicalvisionpermutatorforhyperspectralimageclassification AT yihaoshao spatialspectralhierarchicalvisionpermutatorforhyperspectralimageclassification AT jianjunliu spatialspectralhierarchicalvisionpermutatorforhyperspectralimageclassification AT liangxiao spatialspectralhierarchicalvisionpermutatorforhyperspectralimageclassification |