Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner
Compared with multispectral sensors, hyperspectral sensors obtain images with high- spectral resolution at the cost of spatial resolution, which constrains the further and precise application of hyperspectral images. An intelligent idea to obtain high-resolution hyperspectral images is hyperspectral...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-08-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/16/3226 |
| _version_ | 1797522154310336512 |
|---|---|
| author | Jianhao Gao Jie Li Menghui Jiang |
| author_facet | Jianhao Gao Jie Li Menghui Jiang |
| author_sort | Jianhao Gao |
| collection | DOAJ |
| description | Compared with multispectral sensors, hyperspectral sensors obtain images with high- spectral resolution at the cost of spatial resolution, which constrains the further and precise application of hyperspectral images. An intelligent idea to obtain high-resolution hyperspectral images is hyperspectral and multispectral image fusion. In recent years, many studies have found that deep learning-based fusion methods outperform the traditional fusion methods due to the strong non-linear fitting ability of convolution neural network. However, the function of deep learning-based methods heavily depends on the size and quality of training dataset, constraining the application of deep learning under the situation where training dataset is not available or of low quality. In this paper, we introduce a novel fusion method, which operates in a self-supervised manner, to the task of hyperspectral and multispectral image fusion without training datasets. Our method proposes two constraints constructed by low-resolution hyperspectral images and fake high-resolution hyperspectral images obtained from a simple diffusion method. Several simulation and real-data experiments are conducted with several popular remote sensing hyperspectral data under the condition where training datasets are unavailable. Quantitative and qualitative results indicate that the proposed method outperforms those traditional methods by a large extent. |
| first_indexed | 2024-03-10T08:25:27Z |
| format | Article |
| id | doaj.art-7a81dc5b42894b6c949b1705e8a960cf |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T08:25:27Z |
| publishDate | 2021-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-7a81dc5b42894b6c949b1705e8a960cf2023-11-22T09:34:05ZengMDPI AGRemote Sensing2072-42922021-08-011316322610.3390/rs13163226Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised MannerJianhao Gao0Jie Li1Menghui Jiang2School of Geodesy and Geomatics, Wuhan University, Wuhan 430072, ChinaSchool of Geodesy and Geomatics, Wuhan University, Wuhan 430072, ChinaSchool of Resource and Environmental Sciences, Wuhan University, Wuhan 430072, ChinaCompared with multispectral sensors, hyperspectral sensors obtain images with high- spectral resolution at the cost of spatial resolution, which constrains the further and precise application of hyperspectral images. An intelligent idea to obtain high-resolution hyperspectral images is hyperspectral and multispectral image fusion. In recent years, many studies have found that deep learning-based fusion methods outperform the traditional fusion methods due to the strong non-linear fitting ability of convolution neural network. However, the function of deep learning-based methods heavily depends on the size and quality of training dataset, constraining the application of deep learning under the situation where training dataset is not available or of low quality. In this paper, we introduce a novel fusion method, which operates in a self-supervised manner, to the task of hyperspectral and multispectral image fusion without training datasets. Our method proposes two constraints constructed by low-resolution hyperspectral images and fake high-resolution hyperspectral images obtained from a simple diffusion method. Several simulation and real-data experiments are conducted with several popular remote sensing hyperspectral data under the condition where training datasets are unavailable. Quantitative and qualitative results indicate that the proposed method outperforms those traditional methods by a large extent.https://www.mdpi.com/2072-4292/13/16/3226deep neural networkhyperspectral and multispectral fusionself-supervised optimization |
| spellingShingle | Jianhao Gao Jie Li Menghui Jiang Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner Remote Sensing deep neural network hyperspectral and multispectral fusion self-supervised optimization |
| title | Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner |
| title_full | Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner |
| title_fullStr | Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner |
| title_full_unstemmed | Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner |
| title_short | Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner |
| title_sort | hyperspectral and multispectral image fusion by deep neural network in a self supervised manner |
| topic | deep neural network hyperspectral and multispectral fusion self-supervised optimization |
| url | https://www.mdpi.com/2072-4292/13/16/3226 |
| work_keys_str_mv | AT jianhaogao hyperspectralandmultispectralimagefusionbydeepneuralnetworkinaselfsupervisedmanner AT jieli hyperspectralandmultispectralimagefusionbydeepneuralnetworkinaselfsupervisedmanner AT menghuijiang hyperspectralandmultispectralimagefusionbydeepneuralnetworkinaselfsupervisedmanner |