Multiscale Pixel-Level and Superpixel-Level Method for Hyperspectral Image Classification: Adaptive Attention and Parallel Multi-Hop Graph Convolution
Convolutional neural networks (CNNs) and graph convolutional networks (GCNs) have led to promising advancements in hyperspectral image (HSI) classification; however, traditional CNNs with fixed square convolution kernels are insufficiently flexible to handle irregular structures. Similarly, GCNs tha...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/15/17/4235 |
| _version_ | 1827727896782831616 |
|---|---|
| author | Junru Yin Xuan Liu Ruixia Hou Qiqiang Chen Wei Huang Aiguang Li Peng Wang |
| author_facet | Junru Yin Xuan Liu Ruixia Hou Qiqiang Chen Wei Huang Aiguang Li Peng Wang |
| author_sort | Junru Yin |
| collection | DOAJ |
| description | Convolutional neural networks (CNNs) and graph convolutional networks (GCNs) have led to promising advancements in hyperspectral image (HSI) classification; however, traditional CNNs with fixed square convolution kernels are insufficiently flexible to handle irregular structures. Similarly, GCNs that employ superpixel nodes instead of pixel nodes may overlook pixel-level features; both networks tend to extract features locally and cause loss of multilayer contextual semantic information during feature extraction due to the fixed kernel. To leverage the strengths of CNNs and GCNs, we propose a multiscale pixel-level and superpixel-level (MPAS)-based HSI classification method. The network consists of two sub-networks for extracting multi-level information of HSIs: a multi-scale hybrid spectral–spatial attention convolution branch (HSSAC) and a parallel multi-hop graph convolution branch (MGCN). HSSAC comprehensively captures pixel-level features with different kernel sizes through parallel multi-scale convolution and cross-path fusion to reduce the semantic information loss caused by fixed convolution kernels during feature extraction and learns adjustable weights from the adaptive spectral–spatial attention module (SSAM) to capture pixel-level feature correlations with less computation. MGCN can systematically aggregate multi-hop contextual information to better model HSIs’ spatial background structure using the relationship between parallel multi-hop graph transformation nodes. The proposed MPAS effectively captures multi-layer contextual semantic features by leveraging pixel-level and superpixel-level spectral–spatial information, which improves the performance of the HSI classification task while ensuring computational efficiency. Extensive evaluation experiments on three real-world HSI datasets demonstrate that MPAS outperforms other state-of-the-art networks, demonstrating its superior feature learning capabilities. |
| first_indexed | 2024-03-10T23:14:31Z |
| format | Article |
| id | doaj.art-8b79e8db46564eb79a157fb6b09d2760 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T23:14:31Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-8b79e8db46564eb79a157fb6b09d27602023-11-19T08:46:21ZengMDPI AGRemote Sensing2072-42922023-08-011517423510.3390/rs15174235Multiscale Pixel-Level and Superpixel-Level Method for Hyperspectral Image Classification: Adaptive Attention and Parallel Multi-Hop Graph ConvolutionJunru Yin0Xuan Liu1Ruixia Hou2Qiqiang Chen3Wei Huang4Aiguang Li5Peng Wang6College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, ChinaCollege of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, ChinaResearch Institute of Resource Information Techniques, CAF, Beijing 100091, ChinaCollege of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, ChinaCollege of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, ChinaCollege of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, ChinaConvolutional neural networks (CNNs) and graph convolutional networks (GCNs) have led to promising advancements in hyperspectral image (HSI) classification; however, traditional CNNs with fixed square convolution kernels are insufficiently flexible to handle irregular structures. Similarly, GCNs that employ superpixel nodes instead of pixel nodes may overlook pixel-level features; both networks tend to extract features locally and cause loss of multilayer contextual semantic information during feature extraction due to the fixed kernel. To leverage the strengths of CNNs and GCNs, we propose a multiscale pixel-level and superpixel-level (MPAS)-based HSI classification method. The network consists of two sub-networks for extracting multi-level information of HSIs: a multi-scale hybrid spectral–spatial attention convolution branch (HSSAC) and a parallel multi-hop graph convolution branch (MGCN). HSSAC comprehensively captures pixel-level features with different kernel sizes through parallel multi-scale convolution and cross-path fusion to reduce the semantic information loss caused by fixed convolution kernels during feature extraction and learns adjustable weights from the adaptive spectral–spatial attention module (SSAM) to capture pixel-level feature correlations with less computation. MGCN can systematically aggregate multi-hop contextual information to better model HSIs’ spatial background structure using the relationship between parallel multi-hop graph transformation nodes. The proposed MPAS effectively captures multi-layer contextual semantic features by leveraging pixel-level and superpixel-level spectral–spatial information, which improves the performance of the HSI classification task while ensuring computational efficiency. Extensive evaluation experiments on three real-world HSI datasets demonstrate that MPAS outperforms other state-of-the-art networks, demonstrating its superior feature learning capabilities.https://www.mdpi.com/2072-4292/15/17/4235convolutional neural networks (CNNs)graph convolutional networks (GCNs)hyperspectral image (HSI) classificationattention mechanismmulti-scale features |
| spellingShingle | Junru Yin Xuan Liu Ruixia Hou Qiqiang Chen Wei Huang Aiguang Li Peng Wang Multiscale Pixel-Level and Superpixel-Level Method for Hyperspectral Image Classification: Adaptive Attention and Parallel Multi-Hop Graph Convolution Remote Sensing convolutional neural networks (CNNs) graph convolutional networks (GCNs) hyperspectral image (HSI) classification attention mechanism multi-scale features |
| title | Multiscale Pixel-Level and Superpixel-Level Method for Hyperspectral Image Classification: Adaptive Attention and Parallel Multi-Hop Graph Convolution |
| title_full | Multiscale Pixel-Level and Superpixel-Level Method for Hyperspectral Image Classification: Adaptive Attention and Parallel Multi-Hop Graph Convolution |
| title_fullStr | Multiscale Pixel-Level and Superpixel-Level Method for Hyperspectral Image Classification: Adaptive Attention and Parallel Multi-Hop Graph Convolution |
| title_full_unstemmed | Multiscale Pixel-Level and Superpixel-Level Method for Hyperspectral Image Classification: Adaptive Attention and Parallel Multi-Hop Graph Convolution |
| title_short | Multiscale Pixel-Level and Superpixel-Level Method for Hyperspectral Image Classification: Adaptive Attention and Parallel Multi-Hop Graph Convolution |
| title_sort | multiscale pixel level and superpixel level method for hyperspectral image classification adaptive attention and parallel multi hop graph convolution |
| topic | convolutional neural networks (CNNs) graph convolutional networks (GCNs) hyperspectral image (HSI) classification attention mechanism multi-scale features |
| url | https://www.mdpi.com/2072-4292/15/17/4235 |
| work_keys_str_mv | AT junruyin multiscalepixellevelandsuperpixellevelmethodforhyperspectralimageclassificationadaptiveattentionandparallelmultihopgraphconvolution AT xuanliu multiscalepixellevelandsuperpixellevelmethodforhyperspectralimageclassificationadaptiveattentionandparallelmultihopgraphconvolution AT ruixiahou multiscalepixellevelandsuperpixellevelmethodforhyperspectralimageclassificationadaptiveattentionandparallelmultihopgraphconvolution AT qiqiangchen multiscalepixellevelandsuperpixellevelmethodforhyperspectralimageclassificationadaptiveattentionandparallelmultihopgraphconvolution AT weihuang multiscalepixellevelandsuperpixellevelmethodforhyperspectralimageclassificationadaptiveattentionandparallelmultihopgraphconvolution AT aiguangli multiscalepixellevelandsuperpixellevelmethodforhyperspectralimageclassificationadaptiveattentionandparallelmultihopgraphconvolution AT pengwang multiscalepixellevelandsuperpixellevelmethodforhyperspectralimageclassificationadaptiveattentionandparallelmultihopgraphconvolution |