ESC-PAN: An Efficient CNN Architecture for Image Super-Resolution
Deep Learning models, based on Convolutional Neural Network (CNN) architecture, have proven to be useful and effective in many image processing tasks, and have recently been shown to be effective for image Super-Resolution (SR). Common trends in SR improve the quality of the reconstructed image by i...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10188817/ |
| _version_ | 1797768425895886848 |
|---|---|
| author | Adnan Hamida Motaz Alfarraj Salam A. Zummo |
| author_facet | Adnan Hamida Motaz Alfarraj Salam A. Zummo |
| author_sort | Adnan Hamida |
| collection | DOAJ |
| description | Deep Learning models, based on Convolutional Neural Network (CNN) architecture, have proven to be useful and effective in many image processing tasks, and have recently been shown to be effective for image Super-Resolution (SR). Common trends in SR improve the quality of the reconstructed image by increasing the depth and complexity of the CNN model. While this approach produces superior performance in objective image quality metrics (IQA), such as Peak-Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) index, having the number of parameters in the order of millions sacrifices the practicality of model deployment. This is especially true for applications that require real-time processing, such as online conferencing. In this paper, a CNN-based SR model architecture that integrates an attention mechanism while maintaining low complexity is proposed. The number of parameters of the model is reduced by adopting depthwise-separable convolution (DSC) throughout the model. Multiply-accumulate operations (MACs) are reduced by adopting a late upsampling scheme to operate only on low-dimensional features maps. Experimental results show that the proposed model architecture has better performance in terms of objective IQA metrics, such as PSNR and SSIM, and subjective IQA. This improved performance is achieved at a reduced complexity. We also showcase the scalability of the proposed CNN architecture by increasing the model complexity slightly to gain better desired performance. |
| first_indexed | 2024-03-12T20:53:59Z |
| format | Article |
| id | doaj.art-a127e5b52e6b4049a3335d1ddeb2dea7 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-12T20:53:59Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a127e5b52e6b4049a3335d1ddeb2dea72023-07-31T23:00:13ZengIEEEIEEE Access2169-35362023-01-0111761207613110.1109/ACCESS.2023.329714410188817ESC-PAN: An Efficient CNN Architecture for Image Super-ResolutionAdnan Hamida0https://orcid.org/0000-0002-7209-0960Motaz Alfarraj1https://orcid.org/0000-0002-6052-7221Salam A. Zummo2https://orcid.org/0000-0002-8517-0724Department of Electrical and Computer Engineering, University of Toronto, Toronto, CanadaDepartment of Electrical and Computer Engineering, University of Toronto, Toronto, CanadaDepartment of Electrical and Computer Engineering, University of Toronto, Toronto, CanadaDeep Learning models, based on Convolutional Neural Network (CNN) architecture, have proven to be useful and effective in many image processing tasks, and have recently been shown to be effective for image Super-Resolution (SR). Common trends in SR improve the quality of the reconstructed image by increasing the depth and complexity of the CNN model. While this approach produces superior performance in objective image quality metrics (IQA), such as Peak-Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) index, having the number of parameters in the order of millions sacrifices the practicality of model deployment. This is especially true for applications that require real-time processing, such as online conferencing. In this paper, a CNN-based SR model architecture that integrates an attention mechanism while maintaining low complexity is proposed. The number of parameters of the model is reduced by adopting depthwise-separable convolution (DSC) throughout the model. Multiply-accumulate operations (MACs) are reduced by adopting a late upsampling scheme to operate only on low-dimensional features maps. Experimental results show that the proposed model architecture has better performance in terms of objective IQA metrics, such as PSNR and SSIM, and subjective IQA. This improved performance is achieved at a reduced complexity. We also showcase the scalability of the proposed CNN architecture by increasing the model complexity slightly to gain better desired performance.https://ieeexplore.ieee.org/document/10188817/Real-time image super-resolutiondepth-wise separable convolutionself-calibrated convolutionpixel attentionimage quality assessment |
| spellingShingle | Adnan Hamida Motaz Alfarraj Salam A. Zummo ESC-PAN: An Efficient CNN Architecture for Image Super-Resolution IEEE Access Real-time image super-resolution depth-wise separable convolution self-calibrated convolution pixel attention image quality assessment |
| title | ESC-PAN: An Efficient CNN Architecture for Image Super-Resolution |
| title_full | ESC-PAN: An Efficient CNN Architecture for Image Super-Resolution |
| title_fullStr | ESC-PAN: An Efficient CNN Architecture for Image Super-Resolution |
| title_full_unstemmed | ESC-PAN: An Efficient CNN Architecture for Image Super-Resolution |
| title_short | ESC-PAN: An Efficient CNN Architecture for Image Super-Resolution |
| title_sort | esc pan an efficient cnn architecture for image super resolution |
| topic | Real-time image super-resolution depth-wise separable convolution self-calibrated convolution pixel attention image quality assessment |
| url | https://ieeexplore.ieee.org/document/10188817/ |
| work_keys_str_mv | AT adnanhamida escpananefficientcnnarchitectureforimagesuperresolution AT motazalfarraj escpananefficientcnnarchitectureforimagesuperresolution AT salamazummo escpananefficientcnnarchitectureforimagesuperresolution |