Bag-of-Features-Driven Spectral-Spatial Siamese Neural Network for Hyperspectral Image Classification
Deep learning (DL) exhibits commendable performance in hyperspectral image (HSI) classification because of its powerful feature expression ability. Siamese neural network further improves the performance of DL models by learning similarities within-class and differences between-class from sample pai...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10003978/ |
| _version_ | 1827362072636162048 |
|---|---|
| author | Zhaohui Xue Tianzhi Zhu Yiyang Zhou Mengxue Zhang |
| author_facet | Zhaohui Xue Tianzhi Zhu Yiyang Zhou Mengxue Zhang |
| author_sort | Zhaohui Xue |
| collection | DOAJ |
| description | Deep learning (DL) exhibits commendable performance in hyperspectral image (HSI) classification because of its powerful feature expression ability. Siamese neural network further improves the performance of DL models by learning similarities within-class and differences between-class from sample pairs. However, there are still some limitations in siamese neural network. On the one hand, siamese neural network usually needs a large number of negative pair samples in the training process, leading to computing overhead. On the other hand, current models may lack interpretability because of complex network structure. To overcome the above limitations, we propose a spectral-spatial siamese neural network with bag-of-features (S3BoF) for HSI classification. First, we use a siamese neural network with 3-D and 2-D convolutions to extract the spectral-spatial features. Second, we introduce stop-gradient operation and prediction head structure to make the siamese neural network work without negative pair samples, thus reducing the computational burden. Third, a bag-of-features (BoF) learning module is introduced to enhance the model interpretability and feature representation. Finally, a symmetric loss and a cross entropy loss are respectively used for contrastive learning and classification. Experiments results on four common hyperspectral datasets indicated that S3BoF performs better than the other traditional and state-of-the-art deep learning HSI classification methods in terms of classification accuracy and generalization performance, with improvements in terms of OA around 1.40%–30.01%, 0.27%–8.65%, 0.37%–6.27%, 0.22%–6.64% for Indian Pines, University of Pavia, Salinas, and Yellow River Delta datasets, respectively, under 5% labeled samples per class. |
| first_indexed | 2024-03-08T07:19:42Z |
| format | Article |
| id | doaj.art-c13ce20f1930404098c2d590c1c86d32 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-03-08T07:19:42Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-c13ce20f1930404098c2d590c1c86d322024-02-03T00:01:46ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352023-01-01161085109910.1109/JSTARS.2022.323312510003978Bag-of-Features-Driven Spectral-Spatial Siamese Neural Network for Hyperspectral Image ClassificationZhaohui Xue0https://orcid.org/0000-0001-6253-2967Tianzhi Zhu1https://orcid.org/0000-0002-8294-2274Yiyang Zhou2https://orcid.org/0000-0002-2888-830XMengxue Zhang3https://orcid.org/0000-0002-8587-4334School of Earth Sciences and Engineering, Hohai University, Nanjing, ChinaSchool of Earth Sciences and Engineering, Hohai University, Nanjing, ChinaArtificial Intelligence Laboratory, Hangzhou Hikvision Digital Technology Co., Ltd., Hangzhou, ChinaImage and Signal Processing Group, University of València, València, SpainDeep learning (DL) exhibits commendable performance in hyperspectral image (HSI) classification because of its powerful feature expression ability. Siamese neural network further improves the performance of DL models by learning similarities within-class and differences between-class from sample pairs. However, there are still some limitations in siamese neural network. On the one hand, siamese neural network usually needs a large number of negative pair samples in the training process, leading to computing overhead. On the other hand, current models may lack interpretability because of complex network structure. To overcome the above limitations, we propose a spectral-spatial siamese neural network with bag-of-features (S3BoF) for HSI classification. First, we use a siamese neural network with 3-D and 2-D convolutions to extract the spectral-spatial features. Second, we introduce stop-gradient operation and prediction head structure to make the siamese neural network work without negative pair samples, thus reducing the computational burden. Third, a bag-of-features (BoF) learning module is introduced to enhance the model interpretability and feature representation. Finally, a symmetric loss and a cross entropy loss are respectively used for contrastive learning and classification. Experiments results on four common hyperspectral datasets indicated that S3BoF performs better than the other traditional and state-of-the-art deep learning HSI classification methods in terms of classification accuracy and generalization performance, with improvements in terms of OA around 1.40%–30.01%, 0.27%–8.65%, 0.37%–6.27%, 0.22%–6.64% for Indian Pines, University of Pavia, Salinas, and Yellow River Delta datasets, respectively, under 5% labeled samples per class.https://ieeexplore.ieee.org/document/10003978/Bag-of-features (BoF)deep learning (DL)hyperspectral image (HSI)siamese neural networkspectral-spatial classification |
| spellingShingle | Zhaohui Xue Tianzhi Zhu Yiyang Zhou Mengxue Zhang Bag-of-Features-Driven Spectral-Spatial Siamese Neural Network for Hyperspectral Image Classification IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Bag-of-features (BoF) deep learning (DL) hyperspectral image (HSI) siamese neural network spectral-spatial classification |
| title | Bag-of-Features-Driven Spectral-Spatial Siamese Neural Network for Hyperspectral Image Classification |
| title_full | Bag-of-Features-Driven Spectral-Spatial Siamese Neural Network for Hyperspectral Image Classification |
| title_fullStr | Bag-of-Features-Driven Spectral-Spatial Siamese Neural Network for Hyperspectral Image Classification |
| title_full_unstemmed | Bag-of-Features-Driven Spectral-Spatial Siamese Neural Network for Hyperspectral Image Classification |
| title_short | Bag-of-Features-Driven Spectral-Spatial Siamese Neural Network for Hyperspectral Image Classification |
| title_sort | bag of features driven spectral spatial siamese neural network for hyperspectral image classification |
| topic | Bag-of-features (BoF) deep learning (DL) hyperspectral image (HSI) siamese neural network spectral-spatial classification |
| url | https://ieeexplore.ieee.org/document/10003978/ |
| work_keys_str_mv | AT zhaohuixue bagoffeaturesdrivenspectralspatialsiameseneuralnetworkforhyperspectralimageclassification AT tianzhizhu bagoffeaturesdrivenspectralspatialsiameseneuralnetworkforhyperspectralimageclassification AT yiyangzhou bagoffeaturesdrivenspectralspatialsiameseneuralnetworkforhyperspectralimageclassification AT mengxuezhang bagoffeaturesdrivenspectralspatialsiameseneuralnetworkforhyperspectralimageclassification |