Thin Cloud Removal Fusing Full Spectral and Spatial Features for Sentinel-2 Imagery
Multispectral remote sensing images are widely used for monitoring the globe. Although thin clouds can affect all optical bands, the influences of thin clouds differ with band wavelength. When processing multispectral bands at different resolutions, many methods only remove thin clouds in visible&am...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9910392/ |
| _version_ | 1811344609422344192 |
|---|---|
| author | Jun Li Yuejie Zhang Qinghong Sheng Zhaocong Wu Bo Wang Zhongwen Hu Guanting Shen Michael Schmitt Matthieu Molinier |
| author_facet | Jun Li Yuejie Zhang Qinghong Sheng Zhaocong Wu Bo Wang Zhongwen Hu Guanting Shen Michael Schmitt Matthieu Molinier |
| author_sort | Jun Li |
| collection | DOAJ |
| description | Multispectral remote sensing images are widely used for monitoring the globe. Although thin clouds can affect all optical bands, the influences of thin clouds differ with band wavelength. When processing multispectral bands at different resolutions, many methods only remove thin clouds in visible/near-infrared bands or rescale multiresolution bands to the same resolution and then process them together. The former cannot make full use of multispectral information, and in the latter, the rescaling process will introduce noise. In this article, a deep-learning-based thin cloud removal method that fuses full spectral and spatial features in original Sentinel-2 bands is proposed, named CR4S2. A multi-input and output architecture is designed for better fusing information in all bands and reconstructing the background at original resolutions. In addition, two parallel downsampling residual blocks are designed to transfer features extracted from different depths to the bottom of the network. Experiments were conducted on a new globally distributed Sentinel-2 thin cloud removal dataset called WHUS2-CRv. The results show that the best averaged peak signal-to-noise ratio, structural similarity index measurement, normalized root-mean-square error, and spectral angle mapper of the proposed method over 12 bands in all 20 testing images were 39.55, 0.9443, 0.0245, and 2.5676°, respectively. Compared with baseline methods, the proposed CR4S2 method can better restore not only the spatial features but also spectral features. This indicates that the proposed method is very promising for removing thin clouds in multispectral remote sensing images at different resolutions. |
| first_indexed | 2024-04-13T19:49:45Z |
| format | Article |
| id | doaj.art-1472628bb4464853814c507b619d7365 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-04-13T19:49:45Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-1472628bb4464853814c507b619d73652022-12-22T02:32:34ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352022-01-01158759877510.1109/JSTARS.2022.32118579910392Thin Cloud Removal Fusing Full Spectral and Spatial Features for Sentinel-2 ImageryJun Li0https://orcid.org/0000-0003-0941-8713Yuejie Zhang1Qinghong Sheng2https://orcid.org/0000-0002-7913-2625Zhaocong Wu3https://orcid.org/0000-0003-2435-5538Bo Wang4https://orcid.org/0000-0003-4350-8974Zhongwen Hu5https://orcid.org/0000-0003-2689-3196Guanting Shen6Michael Schmitt7https://orcid.org/0000-0002-0575-2362Matthieu Molinier8https://orcid.org/0000-0002-2656-001XCollege of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaCollege of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaCollege of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaCollege of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaMNR Key Laboratory for Geo-Environmental Monitoring of Great Bay Area, Shenzhen University, Shenzhen, ChinaSchool of Remote Sensing and Information Engineering, Wuhan University, Wuhan, ChinaDepartment of Aerospace Engineering, University of the Bundeswehr Munich, Neubiberg, GermanyVTT Technical Research Centre of Finland, Ltd., Espoo, FinlandMultispectral remote sensing images are widely used for monitoring the globe. Although thin clouds can affect all optical bands, the influences of thin clouds differ with band wavelength. When processing multispectral bands at different resolutions, many methods only remove thin clouds in visible/near-infrared bands or rescale multiresolution bands to the same resolution and then process them together. The former cannot make full use of multispectral information, and in the latter, the rescaling process will introduce noise. In this article, a deep-learning-based thin cloud removal method that fuses full spectral and spatial features in original Sentinel-2 bands is proposed, named CR4S2. A multi-input and output architecture is designed for better fusing information in all bands and reconstructing the background at original resolutions. In addition, two parallel downsampling residual blocks are designed to transfer features extracted from different depths to the bottom of the network. Experiments were conducted on a new globally distributed Sentinel-2 thin cloud removal dataset called WHUS2-CRv. The results show that the best averaged peak signal-to-noise ratio, structural similarity index measurement, normalized root-mean-square error, and spectral angle mapper of the proposed method over 12 bands in all 20 testing images were 39.55, 0.9443, 0.0245, and 2.5676°, respectively. Compared with baseline methods, the proposed CR4S2 method can better restore not only the spatial features but also spectral features. This indicates that the proposed method is very promising for removing thin clouds in multispectral remote sensing images at different resolutions.https://ieeexplore.ieee.org/document/9910392/Deep learning (DL)multifeature fusionparallel downsample residual block (PDRB)Sentinel-2thin cloud removal |
| spellingShingle | Jun Li Yuejie Zhang Qinghong Sheng Zhaocong Wu Bo Wang Zhongwen Hu Guanting Shen Michael Schmitt Matthieu Molinier Thin Cloud Removal Fusing Full Spectral and Spatial Features for Sentinel-2 Imagery IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Deep learning (DL) multifeature fusion parallel downsample residual block (PDRB) Sentinel-2 thin cloud removal |
| title | Thin Cloud Removal Fusing Full Spectral and Spatial Features for Sentinel-2 Imagery |
| title_full | Thin Cloud Removal Fusing Full Spectral and Spatial Features for Sentinel-2 Imagery |
| title_fullStr | Thin Cloud Removal Fusing Full Spectral and Spatial Features for Sentinel-2 Imagery |
| title_full_unstemmed | Thin Cloud Removal Fusing Full Spectral and Spatial Features for Sentinel-2 Imagery |
| title_short | Thin Cloud Removal Fusing Full Spectral and Spatial Features for Sentinel-2 Imagery |
| title_sort | thin cloud removal fusing full spectral and spatial features for sentinel 2 imagery |
| topic | Deep learning (DL) multifeature fusion parallel downsample residual block (PDRB) Sentinel-2 thin cloud removal |
| url | https://ieeexplore.ieee.org/document/9910392/ |
| work_keys_str_mv | AT junli thincloudremovalfusingfullspectralandspatialfeaturesforsentinel2imagery AT yuejiezhang thincloudremovalfusingfullspectralandspatialfeaturesforsentinel2imagery AT qinghongsheng thincloudremovalfusingfullspectralandspatialfeaturesforsentinel2imagery AT zhaocongwu thincloudremovalfusingfullspectralandspatialfeaturesforsentinel2imagery AT bowang thincloudremovalfusingfullspectralandspatialfeaturesforsentinel2imagery AT zhongwenhu thincloudremovalfusingfullspectralandspatialfeaturesforsentinel2imagery AT guantingshen thincloudremovalfusingfullspectralandspatialfeaturesforsentinel2imagery AT michaelschmitt thincloudremovalfusingfullspectralandspatialfeaturesforsentinel2imagery AT matthieumolinier thincloudremovalfusingfullspectralandspatialfeaturesforsentinel2imagery |