A polarization-spectrum fusion framework based on multiscale transform and generative adversarial network for improving water and different vegetation distinguishability
Synthetic Aperture Radar (SAR) and optical image fusion can gain images with abundant spatial, scattering, and spectral information. However, the current fusion methods are still faced with the challenge of spatial distortion, detail blur and inadequate use of polarization information. Thus, this pa...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-09-01
|
| Series: | International Journal of Applied Earth Observations and Geoinformation |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1569843223002923 |
| _version_ | 1797677712431644672 |
|---|---|
| author | Qihao Chen Mengqing Pang Xiuguo Liu Zhengjia Zhang |
| author_facet | Qihao Chen Mengqing Pang Xiuguo Liu Zhengjia Zhang |
| author_sort | Qihao Chen |
| collection | DOAJ |
| description | Synthetic Aperture Radar (SAR) and optical image fusion can gain images with abundant spatial, scattering, and spectral information. However, the current fusion methods are still faced with the challenge of spatial distortion, detail blur and inadequate use of polarization information. Thus, this paper proposes a Sentinel-1/2 images fusion framework based on multiscale transform (MST) and generative adversarial network (GAN) and designs a polarization-spectrum fusion scheme for multi-polarization SAR and multispectral optical images. Firstly, the framework utilizes GAN to fuse low-frequency coefficients of MST to effectively improve the contrast of fusion results. Secondly, based on this framework, the nonsubsampled shearlet transform (NSST) algorithm of MST is selected to decompose input images. The parameter-adaptive pulse coupled neural network (PAPCNN) model is then utilized for fusing high-frequency coefficients from NSST to reduce spatial distortion while effectively preserving the texture details of images. Thirdly, the “polarization-spectrum” fusion scheme of “VV polarization + B band, VH polarization + NIR band and VV/VH + G band” is designed to effectively improve the distinguishability of water bodies and different vegetation. Finally, the subjective and objective evaluation results of our method are significantly superior to other seven methods. In addition, our method is extended to 14 terrestrial biomes. The experimental results show that fused images fully integrate scattering, texture details and spectral information of Sentinel-1/2 images, compared with ESAWorldCover2020 products, which helps to better distinguish various ground objects in different scenarios. |
| first_indexed | 2024-03-11T22:49:13Z |
| format | Article |
| id | doaj.art-dc66d8ca473e47c9a96e1e71dd60e1b6 |
| institution | Directory Open Access Journal |
| issn | 1569-8432 |
| language | English |
| last_indexed | 2024-03-11T22:49:13Z |
| publishDate | 2023-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | International Journal of Applied Earth Observations and Geoinformation |
| spelling | doaj.art-dc66d8ca473e47c9a96e1e71dd60e1b62023-09-22T04:38:16ZengElsevierInternational Journal of Applied Earth Observations and Geoinformation1569-84322023-09-01123103468A polarization-spectrum fusion framework based on multiscale transform and generative adversarial network for improving water and different vegetation distinguishabilityQihao Chen0Mengqing Pang1Xiuguo Liu2Zhengjia Zhang3School of Geography and Information Engineering, China University of Geosciences, PR ChinaSchool of Geography and Information Engineering, China University of Geosciences, PR ChinaCorresponding author.; School of Geography and Information Engineering, China University of Geosciences, PR ChinaSchool of Geography and Information Engineering, China University of Geosciences, PR ChinaSynthetic Aperture Radar (SAR) and optical image fusion can gain images with abundant spatial, scattering, and spectral information. However, the current fusion methods are still faced with the challenge of spatial distortion, detail blur and inadequate use of polarization information. Thus, this paper proposes a Sentinel-1/2 images fusion framework based on multiscale transform (MST) and generative adversarial network (GAN) and designs a polarization-spectrum fusion scheme for multi-polarization SAR and multispectral optical images. Firstly, the framework utilizes GAN to fuse low-frequency coefficients of MST to effectively improve the contrast of fusion results. Secondly, based on this framework, the nonsubsampled shearlet transform (NSST) algorithm of MST is selected to decompose input images. The parameter-adaptive pulse coupled neural network (PAPCNN) model is then utilized for fusing high-frequency coefficients from NSST to reduce spatial distortion while effectively preserving the texture details of images. Thirdly, the “polarization-spectrum” fusion scheme of “VV polarization + B band, VH polarization + NIR band and VV/VH + G band” is designed to effectively improve the distinguishability of water bodies and different vegetation. Finally, the subjective and objective evaluation results of our method are significantly superior to other seven methods. In addition, our method is extended to 14 terrestrial biomes. The experimental results show that fused images fully integrate scattering, texture details and spectral information of Sentinel-1/2 images, compared with ESAWorldCover2020 products, which helps to better distinguish various ground objects in different scenarios.http://www.sciencedirect.com/science/article/pii/S1569843223002923SARImage fusionGenerative adversarial networkNSSTPAPCNN |
| spellingShingle | Qihao Chen Mengqing Pang Xiuguo Liu Zhengjia Zhang A polarization-spectrum fusion framework based on multiscale transform and generative adversarial network for improving water and different vegetation distinguishability International Journal of Applied Earth Observations and Geoinformation SAR Image fusion Generative adversarial network NSST PAPCNN |
| title | A polarization-spectrum fusion framework based on multiscale transform and generative adversarial network for improving water and different vegetation distinguishability |
| title_full | A polarization-spectrum fusion framework based on multiscale transform and generative adversarial network for improving water and different vegetation distinguishability |
| title_fullStr | A polarization-spectrum fusion framework based on multiscale transform and generative adversarial network for improving water and different vegetation distinguishability |
| title_full_unstemmed | A polarization-spectrum fusion framework based on multiscale transform and generative adversarial network for improving water and different vegetation distinguishability |
| title_short | A polarization-spectrum fusion framework based on multiscale transform and generative adversarial network for improving water and different vegetation distinguishability |
| title_sort | polarization spectrum fusion framework based on multiscale transform and generative adversarial network for improving water and different vegetation distinguishability |
| topic | SAR Image fusion Generative adversarial network NSST PAPCNN |
| url | http://www.sciencedirect.com/science/article/pii/S1569843223002923 |
| work_keys_str_mv | AT qihaochen apolarizationspectrumfusionframeworkbasedonmultiscaletransformandgenerativeadversarialnetworkforimprovingwateranddifferentvegetationdistinguishability AT mengqingpang apolarizationspectrumfusionframeworkbasedonmultiscaletransformandgenerativeadversarialnetworkforimprovingwateranddifferentvegetationdistinguishability AT xiuguoliu apolarizationspectrumfusionframeworkbasedonmultiscaletransformandgenerativeadversarialnetworkforimprovingwateranddifferentvegetationdistinguishability AT zhengjiazhang apolarizationspectrumfusionframeworkbasedonmultiscaletransformandgenerativeadversarialnetworkforimprovingwateranddifferentvegetationdistinguishability AT qihaochen polarizationspectrumfusionframeworkbasedonmultiscaletransformandgenerativeadversarialnetworkforimprovingwateranddifferentvegetationdistinguishability AT mengqingpang polarizationspectrumfusionframeworkbasedonmultiscaletransformandgenerativeadversarialnetworkforimprovingwateranddifferentvegetationdistinguishability AT xiuguoliu polarizationspectrumfusionframeworkbasedonmultiscaletransformandgenerativeadversarialnetworkforimprovingwateranddifferentvegetationdistinguishability AT zhengjiazhang polarizationspectrumfusionframeworkbasedonmultiscaletransformandgenerativeadversarialnetworkforimprovingwateranddifferentvegetationdistinguishability |