A Restoration Scheme for Spatial and Spectral Resolution of the Panchromatic Image Using the Convolutional Neural Network
Remote sensing images are the product of information obtained by various sensors, and the higher the resolution of the image, the more information it contains. Therefore, improving the resolution of the remote sensing image is conducive to identify Earth resources from the remote sensing image. In t...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10384696/ |
| _version_ | 1827369186872000512 |
|---|---|
| author | Xin Jin Ling Liu Xiaoxuan Ren Qian Jiang Shin-Jye Lee Jun Zhang Shaowen Yao |
| author_facet | Xin Jin Ling Liu Xiaoxuan Ren Qian Jiang Shin-Jye Lee Jun Zhang Shaowen Yao |
| author_sort | Xin Jin |
| collection | DOAJ |
| description | Remote sensing images are the product of information obtained by various sensors, and the higher the resolution of the image, the more information it contains. Therefore, improving the resolution of the remote sensing image is conducive to identify Earth resources from the remote sensing image. In this article, we present a multiple-branch panchromatic image resolution restoration network based on the convolutional neural network to improve the spatial and spectral resolution of the panchromatic image simultaneously, named MBPRR-Net. Specifically, we adopt a multibranch structure to extract abundant features and utilize a feature channel mixing block to enhance the interaction of adjacent channels between features. Feature aggregation in our method is used to learn more effective features from each branch, and then a cubic filter is utilized to enhance the aggregated features. After feature extraction, we use a recovery architecture to generate the final image. Moreover, we utilize image super-resolution to restore spatial resolution and image colorization to restore the spectral resolution so that we can compare it with some image colorization and super-resolution methods to verify the proposed method. Experiments show that the performance of our method is outstanding in terms of visual effects and objective evaluation metrics compared with some existing excellent image super-resolution and colorization methods. |
| first_indexed | 2024-03-08T09:43:38Z |
| format | Article |
| id | doaj.art-5713837a6644462d82a85e49d342a8ff |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-03-08T09:43:38Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-5713837a6644462d82a85e49d342a8ff2024-01-30T00:00:56ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352024-01-01173379339310.1109/JSTARS.2024.335185410384696A Restoration Scheme for Spatial and Spectral Resolution of the Panchromatic Image Using the Convolutional Neural NetworkXin Jin0https://orcid.org/0000-0003-2211-2006Ling Liu1https://orcid.org/0000-0002-9093-3941Xiaoxuan Ren2https://orcid.org/0009-0006-0569-6731Qian Jiang3https://orcid.org/0000-0003-3097-0721Shin-Jye Lee4https://orcid.org/0000-0003-4265-5016Jun Zhang5https://orcid.org/0009-0001-7519-2882Shaowen Yao6https://orcid.org/0000-0003-1516-4246Engineering Research Center of Cyberspace, Yunnan University, Kunming, ChinaEngineering Research Center of Cyberspace, Yunnan University, Kunming, ChinaEngineering Research Center of Cyberspace, Yunnan University, Kunming, ChinaEngineering Research Center of Cyberspace, Yunnan University, Kunming, ChinaInstitute of Management of Technology, National Yang Ming Chiao Tung University, Hsinchu, TaiwanChina Mobile Communications Group, Yunnan Company Ltd., Kunming, ChinaEngineering Research Center of Cyberspace, Yunnan University, Kunming, ChinaRemote sensing images are the product of information obtained by various sensors, and the higher the resolution of the image, the more information it contains. Therefore, improving the resolution of the remote sensing image is conducive to identify Earth resources from the remote sensing image. In this article, we present a multiple-branch panchromatic image resolution restoration network based on the convolutional neural network to improve the spatial and spectral resolution of the panchromatic image simultaneously, named MBPRR-Net. Specifically, we adopt a multibranch structure to extract abundant features and utilize a feature channel mixing block to enhance the interaction of adjacent channels between features. Feature aggregation in our method is used to learn more effective features from each branch, and then a cubic filter is utilized to enhance the aggregated features. After feature extraction, we use a recovery architecture to generate the final image. Moreover, we utilize image super-resolution to restore spatial resolution and image colorization to restore the spectral resolution so that we can compare it with some image colorization and super-resolution methods to verify the proposed method. Experiments show that the performance of our method is outstanding in terms of visual effects and objective evaluation metrics compared with some existing excellent image super-resolution and colorization methods.https://ieeexplore.ieee.org/document/10384696/Artificial neural networkdeep learningmultispectral (MS) imagepanchromatic (PAN) imageremote sensing image processing |
| spellingShingle | Xin Jin Ling Liu Xiaoxuan Ren Qian Jiang Shin-Jye Lee Jun Zhang Shaowen Yao A Restoration Scheme for Spatial and Spectral Resolution of the Panchromatic Image Using the Convolutional Neural Network IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Artificial neural network deep learning multispectral (MS) image panchromatic (PAN) image remote sensing image processing |
| title | A Restoration Scheme for Spatial and Spectral Resolution of the Panchromatic Image Using the Convolutional Neural Network |
| title_full | A Restoration Scheme for Spatial and Spectral Resolution of the Panchromatic Image Using the Convolutional Neural Network |
| title_fullStr | A Restoration Scheme for Spatial and Spectral Resolution of the Panchromatic Image Using the Convolutional Neural Network |
| title_full_unstemmed | A Restoration Scheme for Spatial and Spectral Resolution of the Panchromatic Image Using the Convolutional Neural Network |
| title_short | A Restoration Scheme for Spatial and Spectral Resolution of the Panchromatic Image Using the Convolutional Neural Network |
| title_sort | restoration scheme for spatial and spectral resolution of the panchromatic image using the convolutional neural network |
| topic | Artificial neural network deep learning multispectral (MS) image panchromatic (PAN) image remote sensing image processing |
| url | https://ieeexplore.ieee.org/document/10384696/ |
| work_keys_str_mv | AT xinjin arestorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT lingliu arestorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT xiaoxuanren arestorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT qianjiang arestorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT shinjyelee arestorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT junzhang arestorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT shaowenyao arestorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT xinjin restorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT lingliu restorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT xiaoxuanren restorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT qianjiang restorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT shinjyelee restorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT junzhang restorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork AT shaowenyao restorationschemeforspatialandspectralresolutionofthepanchromaticimageusingtheconvolutionalneuralnetwork |