SFFNet: Staged Feature Fusion Network of Connecting Convolutional Neural Networks and Graph Convolutional Neural Networks for Hyperspectral Image Classification
The immense representation power of deep learning frameworks has kept them in the spotlight in hyperspectral image (HSI) classification. Graph Convolutional Neural Networks (GCNs) can be used to compensate for the lack of spatial information in Convolutional Neural Networks (CNNs). However, most GCN...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/14/6/2327 |
| _version_ | 1797242198331228160 |
|---|---|
| author | Hao Li Xiaorui Xiong Chaoxian Liu Yong Ma Shan Zeng Yaqin Li |
| author_facet | Hao Li Xiaorui Xiong Chaoxian Liu Yong Ma Shan Zeng Yaqin Li |
| author_sort | Hao Li |
| collection | DOAJ |
| description | The immense representation power of deep learning frameworks has kept them in the spotlight in hyperspectral image (HSI) classification. Graph Convolutional Neural Networks (GCNs) can be used to compensate for the lack of spatial information in Convolutional Neural Networks (CNNs). However, most GCNs construct graph data structures based on pixel points, which requires the construction of neighborhood matrices on all data. Meanwhile, the setting of GCNs to construct similarity relations based on spatial structure is not fully applicable to HSIs. To make the network more compatible with HSIs, we propose a staged feature fusion model called SFFNet, a neural network framework connecting CNN and GCN models. The CNN performs the first stage of feature extraction, assisted by adding neighboring features and overcoming the defects of local convolution; then, the GCN performs the second stage for classification, and the graph data structure is constructed based on spectral similarity, optimizing the original connectivity relationships. In addition, the framework enables the batch training of the GCN by using the extracted spectral features as nodes, which greatly reduces the hardware requirements. The experimental results on three publicly available benchmark hyperspectral datasets show that our proposed framework outperforms other relevant deep learning models, with an overall classification accuracy of over 97%. |
| first_indexed | 2024-04-24T18:35:24Z |
| format | Article |
| id | doaj.art-3c7d87f149184dda910c05643ba2b48f |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-04-24T18:35:24Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-3c7d87f149184dda910c05643ba2b48f2024-03-27T13:19:19ZengMDPI AGApplied Sciences2076-34172024-03-01146232710.3390/app14062327SFFNet: Staged Feature Fusion Network of Connecting Convolutional Neural Networks and Graph Convolutional Neural Networks for Hyperspectral Image ClassificationHao Li0Xiaorui Xiong1Chaoxian Liu2Yong Ma3Shan Zeng4Yaqin Li5School of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430048, ChinaSchool of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430048, ChinaSchool of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430048, ChinaElectronic Information School, Wuhan University, Wuhan 430072, ChinaSchool of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430048, ChinaSchool of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430048, ChinaThe immense representation power of deep learning frameworks has kept them in the spotlight in hyperspectral image (HSI) classification. Graph Convolutional Neural Networks (GCNs) can be used to compensate for the lack of spatial information in Convolutional Neural Networks (CNNs). However, most GCNs construct graph data structures based on pixel points, which requires the construction of neighborhood matrices on all data. Meanwhile, the setting of GCNs to construct similarity relations based on spatial structure is not fully applicable to HSIs. To make the network more compatible with HSIs, we propose a staged feature fusion model called SFFNet, a neural network framework connecting CNN and GCN models. The CNN performs the first stage of feature extraction, assisted by adding neighboring features and overcoming the defects of local convolution; then, the GCN performs the second stage for classification, and the graph data structure is constructed based on spectral similarity, optimizing the original connectivity relationships. In addition, the framework enables the batch training of the GCN by using the extracted spectral features as nodes, which greatly reduces the hardware requirements. The experimental results on three publicly available benchmark hyperspectral datasets show that our proposed framework outperforms other relevant deep learning models, with an overall classification accuracy of over 97%.https://www.mdpi.com/2076-3417/14/6/2327hyperspectral image classificationconvolutional neural networkgraph convolutional neural network |
| spellingShingle | Hao Li Xiaorui Xiong Chaoxian Liu Yong Ma Shan Zeng Yaqin Li SFFNet: Staged Feature Fusion Network of Connecting Convolutional Neural Networks and Graph Convolutional Neural Networks for Hyperspectral Image Classification Applied Sciences hyperspectral image classification convolutional neural network graph convolutional neural network |
| title | SFFNet: Staged Feature Fusion Network of Connecting Convolutional Neural Networks and Graph Convolutional Neural Networks for Hyperspectral Image Classification |
| title_full | SFFNet: Staged Feature Fusion Network of Connecting Convolutional Neural Networks and Graph Convolutional Neural Networks for Hyperspectral Image Classification |
| title_fullStr | SFFNet: Staged Feature Fusion Network of Connecting Convolutional Neural Networks and Graph Convolutional Neural Networks for Hyperspectral Image Classification |
| title_full_unstemmed | SFFNet: Staged Feature Fusion Network of Connecting Convolutional Neural Networks and Graph Convolutional Neural Networks for Hyperspectral Image Classification |
| title_short | SFFNet: Staged Feature Fusion Network of Connecting Convolutional Neural Networks and Graph Convolutional Neural Networks for Hyperspectral Image Classification |
| title_sort | sffnet staged feature fusion network of connecting convolutional neural networks and graph convolutional neural networks for hyperspectral image classification |
| topic | hyperspectral image classification convolutional neural network graph convolutional neural network |
| url | https://www.mdpi.com/2076-3417/14/6/2327 |
| work_keys_str_mv | AT haoli sffnetstagedfeaturefusionnetworkofconnectingconvolutionalneuralnetworksandgraphconvolutionalneuralnetworksforhyperspectralimageclassification AT xiaoruixiong sffnetstagedfeaturefusionnetworkofconnectingconvolutionalneuralnetworksandgraphconvolutionalneuralnetworksforhyperspectralimageclassification AT chaoxianliu sffnetstagedfeaturefusionnetworkofconnectingconvolutionalneuralnetworksandgraphconvolutionalneuralnetworksforhyperspectralimageclassification AT yongma sffnetstagedfeaturefusionnetworkofconnectingconvolutionalneuralnetworksandgraphconvolutionalneuralnetworksforhyperspectralimageclassification AT shanzeng sffnetstagedfeaturefusionnetworkofconnectingconvolutionalneuralnetworksandgraphconvolutionalneuralnetworksforhyperspectralimageclassification AT yaqinli sffnetstagedfeaturefusionnetworkofconnectingconvolutionalneuralnetworksandgraphconvolutionalneuralnetworksforhyperspectralimageclassification |