Denoising Diffusion Probabilistic Model with Adversarial Learning for Remote Sensing Super-Resolution
Single Image Super-Resolution (SISR) for image enhancement enables the generation of high spatial resolution in Remote Sensing (RS) images without incurring additional costs. This approach offers a practical solution to obtain high-resolution RS images, addressing challenges posed by the expense of...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/16/7/1219 |
| _version_ | 1797212000186531840 |
|---|---|
| author | Jialu Sui Qianqian Wu Man-On Pun |
| author_facet | Jialu Sui Qianqian Wu Man-On Pun |
| author_sort | Jialu Sui |
| collection | DOAJ |
| description | Single Image Super-Resolution (SISR) for image enhancement enables the generation of high spatial resolution in Remote Sensing (RS) images without incurring additional costs. This approach offers a practical solution to obtain high-resolution RS images, addressing challenges posed by the expense of acquisition equipment and unpredictable weather conditions. To address the over-smoothing of the previous SISR models, the diffusion model has been incorporated into RS SISR to generate Super-Resolution (SR) images with enhanced textural details. In this paper, we propose a Diffusion model with Adversarial Learning Strategy (DiffALS) to refine the generative capability of the diffusion model. DiffALS integrates an additional Noise Discriminator (ND) into the training process, employing an adversarial learning strategy on the data distribution learning. This ND guides noise prediction by considering the general correspondence between the noisy image in each step, thereby enhancing the diversity of generated data and the detailed texture prediction of the diffusion model. Furthermore, considering that the diffusion model may exhibit suboptimal performance on traditional pixel-level metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM), we showcase the effectiveness of DiffALS through downstream semantic segmentation applications. Extensive experiments demonstrate that the proposed model achieves remarkable accuracy and notable visual enhancements. Compared to other state-of-the-art methods, our model establishes an improvement of 189 for Fréchet Inception Distance (FID) and 0.002 for Learned Perceptual Image Patch Similarity (LPIPS) in a SR dataset, namely Alsat, and achieves improvements of 0.4%, 0.3%, and 0.2% for F1 score, MIoU, and Accuracy, respectively, in a segmentation dataset, namely Vaihingen. |
| first_indexed | 2024-04-24T10:35:25Z |
| format | Article |
| id | doaj.art-389655b9c4ee464b93f38a4d0a1f7e7b |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-04-24T10:35:25Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-389655b9c4ee464b93f38a4d0a1f7e7b2024-04-12T13:25:39ZengMDPI AGRemote Sensing2072-42922024-03-01167121910.3390/rs16071219Denoising Diffusion Probabilistic Model with Adversarial Learning for Remote Sensing Super-ResolutionJialu Sui0Qianqian Wu1Man-On Pun2Future Network of Intelligence Institute, Chinese University of Hong Kong, Shenzhen 518172, ChinaFuture Network of Intelligence Institute, Chinese University of Hong Kong, Shenzhen 518172, ChinaSchool of Science and Engineering, Chinese University of Hong Kong, Shenzhen 518172, ChinaSingle Image Super-Resolution (SISR) for image enhancement enables the generation of high spatial resolution in Remote Sensing (RS) images without incurring additional costs. This approach offers a practical solution to obtain high-resolution RS images, addressing challenges posed by the expense of acquisition equipment and unpredictable weather conditions. To address the over-smoothing of the previous SISR models, the diffusion model has been incorporated into RS SISR to generate Super-Resolution (SR) images with enhanced textural details. In this paper, we propose a Diffusion model with Adversarial Learning Strategy (DiffALS) to refine the generative capability of the diffusion model. DiffALS integrates an additional Noise Discriminator (ND) into the training process, employing an adversarial learning strategy on the data distribution learning. This ND guides noise prediction by considering the general correspondence between the noisy image in each step, thereby enhancing the diversity of generated data and the detailed texture prediction of the diffusion model. Furthermore, considering that the diffusion model may exhibit suboptimal performance on traditional pixel-level metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM), we showcase the effectiveness of DiffALS through downstream semantic segmentation applications. Extensive experiments demonstrate that the proposed model achieves remarkable accuracy and notable visual enhancements. Compared to other state-of-the-art methods, our model establishes an improvement of 189 for Fréchet Inception Distance (FID) and 0.002 for Learned Perceptual Image Patch Similarity (LPIPS) in a SR dataset, namely Alsat, and achieves improvements of 0.4%, 0.3%, and 0.2% for F1 score, MIoU, and Accuracy, respectively, in a segmentation dataset, namely Vaihingen.https://www.mdpi.com/2072-4292/16/7/1219diffusion modelsingle image super-resolutionremote sensingadversarial learning strategy |
| spellingShingle | Jialu Sui Qianqian Wu Man-On Pun Denoising Diffusion Probabilistic Model with Adversarial Learning for Remote Sensing Super-Resolution Remote Sensing diffusion model single image super-resolution remote sensing adversarial learning strategy |
| title | Denoising Diffusion Probabilistic Model with Adversarial Learning for Remote Sensing Super-Resolution |
| title_full | Denoising Diffusion Probabilistic Model with Adversarial Learning for Remote Sensing Super-Resolution |
| title_fullStr | Denoising Diffusion Probabilistic Model with Adversarial Learning for Remote Sensing Super-Resolution |
| title_full_unstemmed | Denoising Diffusion Probabilistic Model with Adversarial Learning for Remote Sensing Super-Resolution |
| title_short | Denoising Diffusion Probabilistic Model with Adversarial Learning for Remote Sensing Super-Resolution |
| title_sort | denoising diffusion probabilistic model with adversarial learning for remote sensing super resolution |
| topic | diffusion model single image super-resolution remote sensing adversarial learning strategy |
| url | https://www.mdpi.com/2072-4292/16/7/1219 |
| work_keys_str_mv | AT jialusui denoisingdiffusionprobabilisticmodelwithadversariallearningforremotesensingsuperresolution AT qianqianwu denoisingdiffusionprobabilisticmodelwithadversariallearningforremotesensingsuperresolution AT manonpun denoisingdiffusionprobabilisticmodelwithadversariallearningforremotesensingsuperresolution |