Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement Operator
Pansharpening aims to fuse the abundant spectral information of multispectral (MS) images and the spatial details of panchromatic (PAN) images, yielding a high-spatial-resolution MS (HRMS) image. Traditional methods only focus on the linear model, ignoring the fact that degradation process is a nonl...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-10-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/20/4062 |
| _version_ | 1797513229511950336 |
|---|---|
| author | Weiwei Huang Yan Zhang Jianwei Zhang Yuhui Zheng |
| author_facet | Weiwei Huang Yan Zhang Jianwei Zhang Yuhui Zheng |
| author_sort | Weiwei Huang |
| collection | DOAJ |
| description | Pansharpening aims to fuse the abundant spectral information of multispectral (MS) images and the spatial details of panchromatic (PAN) images, yielding a high-spatial-resolution MS (HRMS) image. Traditional methods only focus on the linear model, ignoring the fact that degradation process is a nonlinear inverse problem. Due to convolutional neural networks (CNNs) having an extraordinary effect in overcoming the shortcomings of traditional linear models, they have been adapted for pansharpening in the past few years. However, most existing CNN-based methods cannot take full advantage of the structural information of images. To address this problem, a new pansharpening method combining a spatial structure enhancement operator with a CNN architecture is employed in this study. The proposed method uses the Sobel operator as an edge-detection operator to extract abundant high-frequency information from the input PAN and MS images, hence obtaining the abundant spatial features of the images. Moreover, we utilize the CNN to acquire the spatial feature maps, preserving the information in both the spatial and spectral domains. Simulated experiments and real-data experiments demonstrated that our method had excellent performance in both quantitative and visual evaluation. |
| first_indexed | 2024-03-10T06:14:40Z |
| format | Article |
| id | doaj.art-c898109bb7214f74b6b8e8e3ba2fdc0a |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T06:14:40Z |
| publishDate | 2021-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-c898109bb7214f74b6b8e8e3ba2fdc0a2023-11-22T19:53:35ZengMDPI AGRemote Sensing2072-42922021-10-011320406210.3390/rs13204062Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement OperatorWeiwei Huang0Yan Zhang1Jianwei Zhang2Yuhui Zheng3Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, ChinaEngineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, ChinaSchool of Mathematics & Statistics, Nanjing University of Information Science and Technology, Nanjing 210044, ChinaEngineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing 210044, ChinaPansharpening aims to fuse the abundant spectral information of multispectral (MS) images and the spatial details of panchromatic (PAN) images, yielding a high-spatial-resolution MS (HRMS) image. Traditional methods only focus on the linear model, ignoring the fact that degradation process is a nonlinear inverse problem. Due to convolutional neural networks (CNNs) having an extraordinary effect in overcoming the shortcomings of traditional linear models, they have been adapted for pansharpening in the past few years. However, most existing CNN-based methods cannot take full advantage of the structural information of images. To address this problem, a new pansharpening method combining a spatial structure enhancement operator with a CNN architecture is employed in this study. The proposed method uses the Sobel operator as an edge-detection operator to extract abundant high-frequency information from the input PAN and MS images, hence obtaining the abundant spatial features of the images. Moreover, we utilize the CNN to acquire the spatial feature maps, preserving the information in both the spatial and spectral domains. Simulated experiments and real-data experiments demonstrated that our method had excellent performance in both quantitative and visual evaluation.https://www.mdpi.com/2072-4292/13/20/4062pansharpeningconvolutional neural networkspatial structure enhancement |
| spellingShingle | Weiwei Huang Yan Zhang Jianwei Zhang Yuhui Zheng Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement Operator Remote Sensing pansharpening convolutional neural network spatial structure enhancement |
| title | Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement Operator |
| title_full | Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement Operator |
| title_fullStr | Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement Operator |
| title_full_unstemmed | Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement Operator |
| title_short | Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement Operator |
| title_sort | convolutional neural network for pansharpening with spatial structure enhancement operator |
| topic | pansharpening convolutional neural network spatial structure enhancement |
| url | https://www.mdpi.com/2072-4292/13/20/4062 |
| work_keys_str_mv | AT weiweihuang convolutionalneuralnetworkforpansharpeningwithspatialstructureenhancementoperator AT yanzhang convolutionalneuralnetworkforpansharpeningwithspatialstructureenhancementoperator AT jianweizhang convolutionalneuralnetworkforpansharpeningwithspatialstructureenhancementoperator AT yuhuizheng convolutionalneuralnetworkforpansharpeningwithspatialstructureenhancementoperator |