Deep Learning-Based Single-Image Super-Resolution: A Comprehensive Review
High-fidelity information, such as 4K quality videos and photographs, is increasing as high-speed internet access becomes more widespread and less expensive. Even though camera sensors’ performance is constantly improving, artificially enhanced photos and videos created by intelligent ima...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10057379/ |
| _version_ | 1811155624786919424 |
|---|---|
| author | Karansingh Chauhan Shail Nimish Patel Malaram Kumhar Jitendra Bhatia Sudeep Tanwar Innocent Ewean Davidson Thokozile F. Mazibuko Ravi Sharma |
| author_facet | Karansingh Chauhan Shail Nimish Patel Malaram Kumhar Jitendra Bhatia Sudeep Tanwar Innocent Ewean Davidson Thokozile F. Mazibuko Ravi Sharma |
| author_sort | Karansingh Chauhan |
| collection | DOAJ |
| description | High-fidelity information, such as 4K quality videos and photographs, is increasing as high-speed internet access becomes more widespread and less expensive. Even though camera sensors’ performance is constantly improving, artificially enhanced photos and videos created by intelligent image processing algorithms have significantly improved image fidelity in recent years. Single image super-resolution is a class of algorithms that produces a high-resolution image from a given low-resolution image. Since the advent of deep learning a decade ago, this field has made significant strides. This paper presents a comprehensive review of the deep learning assisted single image super-resolution domain including generative adversarial network (GAN) models that discusses the prominent architectures, models used, and their merits and demerits. The reason behind covering the GAN models is that it is been known to perform better than the conventional deep learning methods given the resources and the time. For real-world applications with noise and other issues that can cause low-fidelity super resolution (SR) images, we examine another solution based on GAN model. This GAN model-based technique popularly known as blind super resolution is more resilient. We examined the various super-resolution techniques by varying image scaling factors (i.e., 2x, 3x, 4x) to measure PSNR and SSIM metrics for the different datasets. PSNR across the different datasets covered in the experimental Section shows an average of 14–17 % decrease in the score as we move up the image resolution scale from 2x to 4x. This is observed across all the datasets and for every model mentioned in the experimental Section of the paper. The results also show that blind super-resolution outperforms the conventional deep learning methods and the more complex GAN models. GAN models are complex and preferred when the upscale factor is high, while residual and dense models are recommended for smaller upscaling factors. This paper also discusses the applications of image super-resolution, and finally, the paper is concluded with challenges and future directions. |
| first_indexed | 2024-04-10T04:36:35Z |
| format | Article |
| id | doaj.art-8c81b322512143d88a6d421b5a9192d9 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-10T04:36:35Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-8c81b322512143d88a6d421b5a9192d92023-03-10T00:00:15ZengIEEEIEEE Access2169-35362023-01-0111218112183010.1109/ACCESS.2023.325139610057379Deep Learning-Based Single-Image Super-Resolution: A Comprehensive ReviewKaransingh Chauhan0Shail Nimish Patel1Malaram Kumhar2https://orcid.org/0000-0001-5142-3933Jitendra Bhatia3Sudeep Tanwar4https://orcid.org/0000-0002-1776-4651Innocent Ewean Davidson5Thokozile F. Mazibuko6Ravi Sharma7https://orcid.org/0000-0002-8584-9753Department of Computer Science, Dalhousie University, Halifax, NS, CanadaDepartment of Computer Science and Engineering, Ahmedabad University, Ahmedabad, Gujarat, IndiaDepartment of Computer Science and Engineering, Institute of Technology, Nirma University, Ahmedabad, Gujarat, IndiaDepartment of Computer Science and Engineering, Institute of Technology, Nirma University, Ahmedabad, Gujarat, IndiaDepartment of Computer Science and Engineering, Institute of Technology, Nirma University, Ahmedabad, Gujarat, IndiaDepartment of Electrical, Electronic and Computer Engineering, Cape Peninsula University of Technology, Bellville, South AfricaDepartment of Electrical Power Engineering, Durban University of Technology, Durban, South AfricaCentre for Inter-Disciplinary Research and Innovation, University of Petroleum and Energy Studies, Dehradun, IndiaHigh-fidelity information, such as 4K quality videos and photographs, is increasing as high-speed internet access becomes more widespread and less expensive. Even though camera sensors’ performance is constantly improving, artificially enhanced photos and videos created by intelligent image processing algorithms have significantly improved image fidelity in recent years. Single image super-resolution is a class of algorithms that produces a high-resolution image from a given low-resolution image. Since the advent of deep learning a decade ago, this field has made significant strides. This paper presents a comprehensive review of the deep learning assisted single image super-resolution domain including generative adversarial network (GAN) models that discusses the prominent architectures, models used, and their merits and demerits. The reason behind covering the GAN models is that it is been known to perform better than the conventional deep learning methods given the resources and the time. For real-world applications with noise and other issues that can cause low-fidelity super resolution (SR) images, we examine another solution based on GAN model. This GAN model-based technique popularly known as blind super resolution is more resilient. We examined the various super-resolution techniques by varying image scaling factors (i.e., 2x, 3x, 4x) to measure PSNR and SSIM metrics for the different datasets. PSNR across the different datasets covered in the experimental Section shows an average of 14–17 % decrease in the score as we move up the image resolution scale from 2x to 4x. This is observed across all the datasets and for every model mentioned in the experimental Section of the paper. The results also show that blind super-resolution outperforms the conventional deep learning methods and the more complex GAN models. GAN models are complex and preferred when the upscale factor is high, while residual and dense models are recommended for smaller upscaling factors. This paper also discusses the applications of image super-resolution, and finally, the paper is concluded with challenges and future directions.https://ieeexplore.ieee.org/document/10057379/Image super-resolutiondeep learningconvolutional neural networkgenerative adversarial network |
| spellingShingle | Karansingh Chauhan Shail Nimish Patel Malaram Kumhar Jitendra Bhatia Sudeep Tanwar Innocent Ewean Davidson Thokozile F. Mazibuko Ravi Sharma Deep Learning-Based Single-Image Super-Resolution: A Comprehensive Review IEEE Access Image super-resolution deep learning convolutional neural network generative adversarial network |
| title | Deep Learning-Based Single-Image Super-Resolution: A Comprehensive Review |
| title_full | Deep Learning-Based Single-Image Super-Resolution: A Comprehensive Review |
| title_fullStr | Deep Learning-Based Single-Image Super-Resolution: A Comprehensive Review |
| title_full_unstemmed | Deep Learning-Based Single-Image Super-Resolution: A Comprehensive Review |
| title_short | Deep Learning-Based Single-Image Super-Resolution: A Comprehensive Review |
| title_sort | deep learning based single image super resolution a comprehensive review |
| topic | Image super-resolution deep learning convolutional neural network generative adversarial network |
| url | https://ieeexplore.ieee.org/document/10057379/ |
| work_keys_str_mv | AT karansinghchauhan deeplearningbasedsingleimagesuperresolutionacomprehensivereview AT shailnimishpatel deeplearningbasedsingleimagesuperresolutionacomprehensivereview AT malaramkumhar deeplearningbasedsingleimagesuperresolutionacomprehensivereview AT jitendrabhatia deeplearningbasedsingleimagesuperresolutionacomprehensivereview AT sudeeptanwar deeplearningbasedsingleimagesuperresolutionacomprehensivereview AT innocenteweandavidson deeplearningbasedsingleimagesuperresolutionacomprehensivereview AT thokozilefmazibuko deeplearningbasedsingleimagesuperresolutionacomprehensivereview AT ravisharma deeplearningbasedsingleimagesuperresolutionacomprehensivereview |