Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images
Single Image Super-Resolution (SISR) using Convolutional Neural Networks (CNNs) for many applications in supervised manner has resulted in significant improvement in state-of-the-art performance. Such supervised models achieve remarkable accuracy; albeit their poor generalization ability for real-wo...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9954406/ |
| _version_ | 1798016075041865728 |
|---|---|
| author | Kalpesh Prajapati Vishal Chudasama Heena Patel Anjali Sarvaiya Kishor Upla Kiran Raja Raghavendra Ramachandra Christoph Busch |
| author_facet | Kalpesh Prajapati Vishal Chudasama Heena Patel Anjali Sarvaiya Kishor Upla Kiran Raja Raghavendra Ramachandra Christoph Busch |
| author_sort | Kalpesh Prajapati |
| collection | DOAJ |
| description | Single Image Super-Resolution (SISR) using Convolutional Neural Networks (CNNs) for many applications in supervised manner has resulted in significant improvement in state-of-the-art performance. Such supervised models achieve remarkable accuracy; albeit their poor generalization ability for real-world Low-Resolution (LR) images. Supervised training in many SR works involves synthetically generated LR images from its corresponding High-Resolution (HR) images. As the distribution of such LR observation is relatively different from that of real LR image, the supervised training in SISR task results in a degradation when applied on real-world data. SISR has been scaled to real-world data recently by posing the unsupervised problem into a supervised one through learning the distribution of noisy LR observation first, following which supervised training is performed to obtain the SR image. It therefore involves two steps where the accuracy of SR image relies on how closely the LR’s distribution is learnt in the first step. In this work, we overcome such limitation by introducing unsupervised denoising network to transform real noisy LR image to clean image and then pre-trained SR network is utilised to increase the spatial resolution of cleaned LR image to generate SR image. Thus, instead of evaluating the denoised image in LR space to train the denoising network, we inspect the denoised image in SR space which allows to overcome the SR network’s generalization problem. The proposed Unsupervised Denoising framework for Super-Resolution (UDSR) is validated on real-world datasets (NTIRE-2020 Real-World SR Challenge validation and testing dataset (Track-1)) by comparing it with many recent unsupervised SISR methods. The performance of denoising and SR networks is superior in terms of various perceptual indices such as Perceptual Index (PI) and Ma Score in addition to numerous non-references metrics. |
| first_indexed | 2024-04-11T15:45:00Z |
| format | Article |
| id | doaj.art-a64e6bc16fe24bffb541e7a2ea45c470 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T15:45:00Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a64e6bc16fe24bffb541e7a2ea45c4702022-12-22T04:15:37ZengIEEEIEEE Access2169-35362022-01-011012232912234610.1109/ACCESS.2022.32231019954406Unsupervised Denoising for Super-Resolution (UDSR) of Real-World ImagesKalpesh Prajapati0https://orcid.org/0000-0002-0804-5102Vishal Chudasama1https://orcid.org/0000-0002-3727-5484Heena Patel2Anjali Sarvaiya3Kishor Upla4https://orcid.org/0000-0001-6306-0682Kiran Raja5https://orcid.org/0000-0002-9489-5161Raghavendra Ramachandra6https://orcid.org/0000-0003-0484-3956Christoph Busch7https://orcid.org/0000-0002-9159-2923Sardar Vallabhbhai National Institute of Technology (SVNIT), Surat, IndiaSardar Vallabhbhai National Institute of Technology (SVNIT), Surat, IndiaSardar Vallabhbhai National Institute of Technology (SVNIT), Surat, IndiaSardar Vallabhbhai National Institute of Technology (SVNIT), Surat, IndiaSardar Vallabhbhai National Institute of Technology (SVNIT), Surat, IndiaDepartment of Computer Science, Norwegian University of Science and Technology (NTNU), Gjøvik, NorwayDepartment of Information Security and Communication Technology, Norwegian University of Science and Technology (NTNU), Gjøvik, NorwayDepartment of Information Security and Communication Technology, Norwegian University of Science and Technology (NTNU), Gjøvik, NorwaySingle Image Super-Resolution (SISR) using Convolutional Neural Networks (CNNs) for many applications in supervised manner has resulted in significant improvement in state-of-the-art performance. Such supervised models achieve remarkable accuracy; albeit their poor generalization ability for real-world Low-Resolution (LR) images. Supervised training in many SR works involves synthetically generated LR images from its corresponding High-Resolution (HR) images. As the distribution of such LR observation is relatively different from that of real LR image, the supervised training in SISR task results in a degradation when applied on real-world data. SISR has been scaled to real-world data recently by posing the unsupervised problem into a supervised one through learning the distribution of noisy LR observation first, following which supervised training is performed to obtain the SR image. It therefore involves two steps where the accuracy of SR image relies on how closely the LR’s distribution is learnt in the first step. In this work, we overcome such limitation by introducing unsupervised denoising network to transform real noisy LR image to clean image and then pre-trained SR network is utilised to increase the spatial resolution of cleaned LR image to generate SR image. Thus, instead of evaluating the denoised image in LR space to train the denoising network, we inspect the denoised image in SR space which allows to overcome the SR network’s generalization problem. The proposed Unsupervised Denoising framework for Super-Resolution (UDSR) is validated on real-world datasets (NTIRE-2020 Real-World SR Challenge validation and testing dataset (Track-1)) by comparing it with many recent unsupervised SISR methods. The performance of denoising and SR networks is superior in terms of various perceptual indices such as Perceptual Index (PI) and Ma Score in addition to numerous non-references metrics.https://ieeexplore.ieee.org/document/9954406/Convolutional neural networkgenerative adversarial networkimage enhancementimage restorationsingle-image super-resolutionunsupervised learning |
| spellingShingle | Kalpesh Prajapati Vishal Chudasama Heena Patel Anjali Sarvaiya Kishor Upla Kiran Raja Raghavendra Ramachandra Christoph Busch Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images IEEE Access Convolutional neural network generative adversarial network image enhancement image restoration single-image super-resolution unsupervised learning |
| title | Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images |
| title_full | Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images |
| title_fullStr | Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images |
| title_full_unstemmed | Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images |
| title_short | Unsupervised Denoising for Super-Resolution (UDSR) of Real-World Images |
| title_sort | unsupervised denoising for super resolution udsr of real world images |
| topic | Convolutional neural network generative adversarial network image enhancement image restoration single-image super-resolution unsupervised learning |
| url | https://ieeexplore.ieee.org/document/9954406/ |
| work_keys_str_mv | AT kalpeshprajapati unsuperviseddenoisingforsuperresolutionudsrofrealworldimages AT vishalchudasama unsuperviseddenoisingforsuperresolutionudsrofrealworldimages AT heenapatel unsuperviseddenoisingforsuperresolutionudsrofrealworldimages AT anjalisarvaiya unsuperviseddenoisingforsuperresolutionudsrofrealworldimages AT kishorupla unsuperviseddenoisingforsuperresolutionudsrofrealworldimages AT kiranraja unsuperviseddenoisingforsuperresolutionudsrofrealworldimages AT raghavendraramachandra unsuperviseddenoisingforsuperresolutionudsrofrealworldimages AT christophbusch unsuperviseddenoisingforsuperresolutionudsrofrealworldimages |