Attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification
Convolutional neural networks (CNNs) for hyperspectral image (HSI) classification have generated good progress. Meanwhile, graph convolutional networks (GCNs) have also attracted considerable attention by using unlabeled data, broadly and explicitly exploiting correlations between adjacent parcels....
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/172259 |
| _version_ | 1811691220041203712 |
|---|---|
| author | Zhou, Hao Luo, Fulin Zhuang, Huiping Weng, Zhenyu Gong, Xiuwen Lin, Zhiping |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Zhou, Hao Luo, Fulin Zhuang, Huiping Weng, Zhenyu Gong, Xiuwen Lin, Zhiping |
| author_sort | Zhou, Hao |
| collection | NTU |
| description | Convolutional neural networks (CNNs) for hyperspectral image (HSI) classification have generated good progress. Meanwhile, graph convolutional networks (GCNs) have also attracted considerable attention by using unlabeled data, broadly and explicitly exploiting correlations between adjacent parcels. However, the CNN with a fixed square convolution kernel is not flexible enough to deal with irregular patterns, while the GCN using the superpixel to reduce the number of nodes will lose the pixel-level features, and the features from the two networks are always partial. In this article, to make good use of the advantages of CNN and GCN, we propose a novel multiple feature fusion model termed attention multihop graph and multiscale convolutional fusion network (AMGCFN), which includes two subnetworks of multiscale fully CNN and multihop GCN to extract the multilevel information of HSI. Specifically, the multiscale fully CNN aims to comprehensively capture pixel-level features with different kernel sizes, and a multihead attention fusion module (MAFM) is used to fuse the multiscale pixel-level features. The multihop GCN systematically aggregates the multihop contextual information by applying multihop graphs on different layers to transform the relationships between nodes, and an MAFM is adopted to combine the multihop features. Finally, we design a cross-attention fusion module (CAFM) to adaptively fuse the features of two subnetworks. The AMGCFN makes full use of multiscale convolution and multihop graph features, which is conducive to the learning of multilevel contextual semantic features. Experimental results on three benchmark HSI datasets show that the AMGCFN has a better performance than a few state-of-the-art methods. Code: https://github.com/EdwardHaoz/IEEE_TGRS_AMGCFN. |
| first_indexed | 2024-10-01T06:16:25Z |
| format | Journal Article |
| id | ntu-10356/172259 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:16:25Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | ntu-10356/1722592023-12-04T06:00:12Z Attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification Zhou, Hao Luo, Fulin Zhuang, Huiping Weng, Zhenyu Gong, Xiuwen Lin, Zhiping School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Attention Fusion Convolutional Neural Network Convolutional neural networks (CNNs) for hyperspectral image (HSI) classification have generated good progress. Meanwhile, graph convolutional networks (GCNs) have also attracted considerable attention by using unlabeled data, broadly and explicitly exploiting correlations between adjacent parcels. However, the CNN with a fixed square convolution kernel is not flexible enough to deal with irregular patterns, while the GCN using the superpixel to reduce the number of nodes will lose the pixel-level features, and the features from the two networks are always partial. In this article, to make good use of the advantages of CNN and GCN, we propose a novel multiple feature fusion model termed attention multihop graph and multiscale convolutional fusion network (AMGCFN), which includes two subnetworks of multiscale fully CNN and multihop GCN to extract the multilevel information of HSI. Specifically, the multiscale fully CNN aims to comprehensively capture pixel-level features with different kernel sizes, and a multihead attention fusion module (MAFM) is used to fuse the multiscale pixel-level features. The multihop GCN systematically aggregates the multihop contextual information by applying multihop graphs on different layers to transform the relationships between nodes, and an MAFM is adopted to combine the multihop features. Finally, we design a cross-attention fusion module (CAFM) to adaptively fuse the features of two subnetworks. The AMGCFN makes full use of multiscale convolution and multihop graph features, which is conducive to the learning of multilevel contextual semantic features. Experimental results on three benchmark HSI datasets show that the AMGCFN has a better performance than a few state-of-the-art methods. Code: https://github.com/EdwardHaoz/IEEE_TGRS_AMGCFN. This work was supported in part by the National Natural Science Foundation of China under Grant 62071340 and Grant 61801336 and in part by the Natural Science Foundation of Chongqing under Grant CSTB2022NSCQ-MSX0452. 2023-12-04T06:00:12Z 2023-12-04T06:00:12Z 2023 Journal Article Zhou, H., Luo, F., Zhuang, H., Weng, Z., Gong, X. & Lin, Z. (2023). Attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification. IEEE Transactions On Geoscience and Remote Sensing, 61, 3265879-. https://dx.doi.org/10.1109/TGRS.2023.3265879 0196-2892 https://hdl.handle.net/10356/172259 10.1109/TGRS.2023.3265879 2-s2.0-85153329307 61 3265879 en IEEE Transactions on Geoscience and Remote Sensing © 2023 IEEE. All rights reserved. |
| spellingShingle | Engineering::Electrical and electronic engineering Attention Fusion Convolutional Neural Network Zhou, Hao Luo, Fulin Zhuang, Huiping Weng, Zhenyu Gong, Xiuwen Lin, Zhiping Attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification |
| title | Attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification |
| title_full | Attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification |
| title_fullStr | Attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification |
| title_full_unstemmed | Attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification |
| title_short | Attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification |
| title_sort | attention multihop graph and multiscale convolutional fusion network for hyperspectral image classification |
| topic | Engineering::Electrical and electronic engineering Attention Fusion Convolutional Neural Network |
| url | https://hdl.handle.net/10356/172259 |
| work_keys_str_mv | AT zhouhao attentionmultihopgraphandmultiscaleconvolutionalfusionnetworkforhyperspectralimageclassification AT luofulin attentionmultihopgraphandmultiscaleconvolutionalfusionnetworkforhyperspectralimageclassification AT zhuanghuiping attentionmultihopgraphandmultiscaleconvolutionalfusionnetworkforhyperspectralimageclassification AT wengzhenyu attentionmultihopgraphandmultiscaleconvolutionalfusionnetworkforhyperspectralimageclassification AT gongxiuwen attentionmultihopgraphandmultiscaleconvolutionalfusionnetworkforhyperspectralimageclassification AT linzhiping attentionmultihopgraphandmultiscaleconvolutionalfusionnetworkforhyperspectralimageclassification |