Spatial Transformer Generative Adversarial Network for Robust Image Super-Resolution
Recently, there have been significant advances in image super-resolution based on generative adversarial networks (GANs) to achieve breakthroughs in generating more images with high subjective quality. However, there are remaining challenges needs to be met, such as simultaneously recovering the fin...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8933156/ |
| _version_ | 1818412658512101376 |
|---|---|
| author | Hossam M. Kasem Kwok-Wai Hung Jianmin Jiang |
| author_facet | Hossam M. Kasem Kwok-Wai Hung Jianmin Jiang |
| author_sort | Hossam M. Kasem |
| collection | DOAJ |
| description | Recently, there have been significant advances in image super-resolution based on generative adversarial networks (GANs) to achieve breakthroughs in generating more images with high subjective quality. However, there are remaining challenges needs to be met, such as simultaneously recovering the finer texture details for large upscaling factors and mitigating the geometric transformation effects. In this paper, we propose a novel robust super-resolution GAN (i.e. namely RSR-GAN) which can simultaneously perform both the geometric transformation and recovering the finer texture details. Specifically, since the performance of the generator depends on the discreminator, we propose a novel discriminator design by incorporating the spatial transformer module with residual learning to improve the discrimination of fake and true images through removing the geometric noise, in order to enhance the super-resolution of geometric corrected images. Finally, to further improve the perceptual quality, we introduce an additional DCT loss term into the existing loss function. Extensive experiments, measured by both PSNR and SSIM measurements, show that our proposed method achieves a high level of robustness against a number of geometric transformations, including rotation, translation, a combination of rotation and scaling effects, and a cobmination of rotaion, transalation and scaling effects. Benchmarked by the existing state-of-the-arts SR methods, our proposed delivers superior performances on a wide range of datasets which are publicly available and widely adopted across research communities. |
| first_indexed | 2024-12-14T10:50:49Z |
| format | Article |
| id | doaj.art-03e7dc52ee504515ab24c71ec84af694 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-14T10:50:49Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-03e7dc52ee504515ab24c71ec84af6942022-12-21T23:05:15ZengIEEEIEEE Access2169-35362019-01-01718299318300910.1109/ACCESS.2019.29599408933156Spatial Transformer Generative Adversarial Network for Robust Image Super-ResolutionHossam M. Kasem0https://orcid.org/0000-0001-8680-9609Kwok-Wai Hung1https://orcid.org/0000-0002-1665-3669Jianmin Jiang2https://orcid.org/0000-0002-7576-3999Research Institute for Future Media Computing, College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, ChinaResearch Institute for Future Media Computing, College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, ChinaResearch Institute for Future Media Computing, College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, ChinaRecently, there have been significant advances in image super-resolution based on generative adversarial networks (GANs) to achieve breakthroughs in generating more images with high subjective quality. However, there are remaining challenges needs to be met, such as simultaneously recovering the finer texture details for large upscaling factors and mitigating the geometric transformation effects. In this paper, we propose a novel robust super-resolution GAN (i.e. namely RSR-GAN) which can simultaneously perform both the geometric transformation and recovering the finer texture details. Specifically, since the performance of the generator depends on the discreminator, we propose a novel discriminator design by incorporating the spatial transformer module with residual learning to improve the discrimination of fake and true images through removing the geometric noise, in order to enhance the super-resolution of geometric corrected images. Finally, to further improve the perceptual quality, we introduce an additional DCT loss term into the existing loss function. Extensive experiments, measured by both PSNR and SSIM measurements, show that our proposed method achieves a high level of robustness against a number of geometric transformations, including rotation, translation, a combination of rotation and scaling effects, and a cobmination of rotaion, transalation and scaling effects. Benchmarked by the existing state-of-the-arts SR methods, our proposed delivers superior performances on a wide range of datasets which are publicly available and widely adopted across research communities.https://ieeexplore.ieee.org/document/8933156/Super-resolutiongenerative adversarial networksspatial transformer networkrobust image super-resolutionrobust generative adversarial network |
| spellingShingle | Hossam M. Kasem Kwok-Wai Hung Jianmin Jiang Spatial Transformer Generative Adversarial Network for Robust Image Super-Resolution IEEE Access Super-resolution generative adversarial networks spatial transformer network robust image super-resolution robust generative adversarial network |
| title | Spatial Transformer Generative Adversarial Network for Robust Image Super-Resolution |
| title_full | Spatial Transformer Generative Adversarial Network for Robust Image Super-Resolution |
| title_fullStr | Spatial Transformer Generative Adversarial Network for Robust Image Super-Resolution |
| title_full_unstemmed | Spatial Transformer Generative Adversarial Network for Robust Image Super-Resolution |
| title_short | Spatial Transformer Generative Adversarial Network for Robust Image Super-Resolution |
| title_sort | spatial transformer generative adversarial network for robust image super resolution |
| topic | Super-resolution generative adversarial networks spatial transformer network robust image super-resolution robust generative adversarial network |
| url | https://ieeexplore.ieee.org/document/8933156/ |
| work_keys_str_mv | AT hossammkasem spatialtransformergenerativeadversarialnetworkforrobustimagesuperresolution AT kwokwaihung spatialtransformergenerativeadversarialnetworkforrobustimagesuperresolution AT jianminjiang spatialtransformergenerativeadversarialnetworkforrobustimagesuperresolution |