SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction
Accurate hyperspectral remote sensing information is essential for feature identification and detection. Nevertheless, the hyperspectral imaging mechanism poses challenges in balancing the trade-off between spatial and spectral resolution. Hardware improvements are cost-intensive and depend on stric...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/7/3728 |
| _version_ | 1797606993974788096 |
|---|---|
| author | Lingxuan Zhu Jiaji Wu Wang Biao Yi Liao Dandan Gu |
| author_facet | Lingxuan Zhu Jiaji Wu Wang Biao Yi Liao Dandan Gu |
| author_sort | Lingxuan Zhu |
| collection | DOAJ |
| description | Accurate hyperspectral remote sensing information is essential for feature identification and detection. Nevertheless, the hyperspectral imaging mechanism poses challenges in balancing the trade-off between spatial and spectral resolution. Hardware improvements are cost-intensive and depend on strict environmental conditions and extra equipment. Recent spectral imaging methods have attempted to directly reconstruct hyperspectral information from widely available multispectral images. However, fixed mapping approaches used in previous spectral reconstruction models limit their reconstruction quality and generalizability, especially dealing with missing or contaminated bands. Moreover, data-hungry issues plague increasingly complex data-driven spectral reconstruction methods. This paper proposes SpectralMAE, a novel spectral reconstruction model that can take arbitrary combinations of bands as input and improve the utilization of data sources. In contrast to previous spectral reconstruction techniques, SpectralMAE explores the application of a self-supervised learning paradigm and proposes a masked autoencoder architecture for spectral dimensions. To further enhance the performance for specific sensor inputs, we propose a training strategy by combining random masking pre-training and fixed masking fine-tuning. Empirical evaluations on five remote sensing datasets demonstrate that SpectralMAE outperforms state-of-the-art methods in both qualitative and quantitative metrics. |
| first_indexed | 2024-03-11T05:25:02Z |
| format | Article |
| id | doaj.art-aa8ccb527c8b444ca965c88b1522d9ab |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T05:25:02Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-aa8ccb527c8b444ca965c88b1522d9ab2023-11-17T17:36:53ZengMDPI AGSensors1424-82202023-04-01237372810.3390/s23073728SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image ReconstructionLingxuan Zhu0Jiaji Wu1Wang Biao2Yi Liao3Dandan Gu4School of Electronic Engineering, Xidian University, Xi’an 710071, ChinaSchool of Electronic Engineering, Xidian University, Xi’an 710071, ChinaSchool of Electronic Engineering, Xidian University, Xi’an 710071, ChinaNational Key Laboratory of Scattering and Radiation, Shanghai 200438, ChinaNational Key Laboratory of Scattering and Radiation, Shanghai 200438, ChinaAccurate hyperspectral remote sensing information is essential for feature identification and detection. Nevertheless, the hyperspectral imaging mechanism poses challenges in balancing the trade-off between spatial and spectral resolution. Hardware improvements are cost-intensive and depend on strict environmental conditions and extra equipment. Recent spectral imaging methods have attempted to directly reconstruct hyperspectral information from widely available multispectral images. However, fixed mapping approaches used in previous spectral reconstruction models limit their reconstruction quality and generalizability, especially dealing with missing or contaminated bands. Moreover, data-hungry issues plague increasingly complex data-driven spectral reconstruction methods. This paper proposes SpectralMAE, a novel spectral reconstruction model that can take arbitrary combinations of bands as input and improve the utilization of data sources. In contrast to previous spectral reconstruction techniques, SpectralMAE explores the application of a self-supervised learning paradigm and proposes a masked autoencoder architecture for spectral dimensions. To further enhance the performance for specific sensor inputs, we propose a training strategy by combining random masking pre-training and fixed masking fine-tuning. Empirical evaluations on five remote sensing datasets demonstrate that SpectralMAE outperforms state-of-the-art methods in both qualitative and quantitative metrics.https://www.mdpi.com/1424-8220/23/7/3728spectral reconstructionhyperspectral imagingmasked autoencoderself-supervised learningtransformer |
| spellingShingle | Lingxuan Zhu Jiaji Wu Wang Biao Yi Liao Dandan Gu SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction Sensors spectral reconstruction hyperspectral imaging masked autoencoder self-supervised learning transformer |
| title | SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction |
| title_full | SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction |
| title_fullStr | SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction |
| title_full_unstemmed | SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction |
| title_short | SpectralMAE: Spectral Masked Autoencoder for Hyperspectral Remote Sensing Image Reconstruction |
| title_sort | spectralmae spectral masked autoencoder for hyperspectral remote sensing image reconstruction |
| topic | spectral reconstruction hyperspectral imaging masked autoencoder self-supervised learning transformer |
| url | https://www.mdpi.com/1424-8220/23/7/3728 |
| work_keys_str_mv | AT lingxuanzhu spectralmaespectralmaskedautoencoderforhyperspectralremotesensingimagereconstruction AT jiajiwu spectralmaespectralmaskedautoencoderforhyperspectralremotesensingimagereconstruction AT wangbiao spectralmaespectralmaskedautoencoderforhyperspectralremotesensingimagereconstruction AT yiliao spectralmaespectralmaskedautoencoderforhyperspectralremotesensingimagereconstruction AT dandangu spectralmaespectralmaskedautoencoderforhyperspectralremotesensingimagereconstruction |