A Dynamic Convolution Kernel Generation Method Based on Regularized Pattern for Image Super-Resolution
Image super-resolution aims to reconstruct a high-resolution image from its low-resolution counterparts. Conventional image super-resolution approaches share the same spatial convolution kernel for the whole image in the upscaling modules, which neglect the specificity of content information in diff...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/11/4231 |
| _version_ | 1797491601701863424 |
|---|---|
| author | Hesen Feng Lihong Ma Jing Tian |
| author_facet | Hesen Feng Lihong Ma Jing Tian |
| author_sort | Hesen Feng |
| collection | DOAJ |
| description | Image super-resolution aims to reconstruct a high-resolution image from its low-resolution counterparts. Conventional image super-resolution approaches share the same spatial convolution kernel for the whole image in the upscaling modules, which neglect the specificity of content information in different positions of the image. In view of this, this paper proposes a regularized pattern method to represent spatially variant structural features in an image and further exploits a dynamic convolution kernel generation method to match the regularized pattern and improve image reconstruction performance. To be more specific, first, the proposed approach extracts features from low-resolution images using a self-organizing feature mapping network to construct regularized patterns (RP), which describe different contents at different locations. Second, the meta-learning mechanism based on the regularized pattern predicts the weights of the convolution kernels that match the regularized pattern for each different location; therefore, it generates different upscaling functions for images with different content. Extensive experiments are conducted using the benchmark datasets Set5, Set14, B100, Urban100, and Manga109 to demonstrate that the proposed approach outperforms the state-of-the-art super-resolution approaches in terms of both PSNR and SSIM performance. |
| first_indexed | 2024-03-10T00:51:39Z |
| format | Article |
| id | doaj.art-2cec638d698648fdbebe12546f5b29f6 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T00:51:39Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-2cec638d698648fdbebe12546f5b29f62023-11-23T14:50:40ZengMDPI AGSensors1424-82202022-06-012211423110.3390/s22114231A Dynamic Convolution Kernel Generation Method Based on Regularized Pattern for Image Super-ResolutionHesen Feng0Lihong Ma1Jing Tian2School of Electronics & Information Engineering, South China University of Technology, Guangzhou 510640, ChinaSchool of Electronics & Information Engineering, South China University of Technology, Guangzhou 510640, ChinaInstitute of Systems Science, National University of Singapore, Singapore 119615, SingaporeImage super-resolution aims to reconstruct a high-resolution image from its low-resolution counterparts. Conventional image super-resolution approaches share the same spatial convolution kernel for the whole image in the upscaling modules, which neglect the specificity of content information in different positions of the image. In view of this, this paper proposes a regularized pattern method to represent spatially variant structural features in an image and further exploits a dynamic convolution kernel generation method to match the regularized pattern and improve image reconstruction performance. To be more specific, first, the proposed approach extracts features from low-resolution images using a self-organizing feature mapping network to construct regularized patterns (RP), which describe different contents at different locations. Second, the meta-learning mechanism based on the regularized pattern predicts the weights of the convolution kernels that match the regularized pattern for each different location; therefore, it generates different upscaling functions for images with different content. Extensive experiments are conducted using the benchmark datasets Set5, Set14, B100, Urban100, and Manga109 to demonstrate that the proposed approach outperforms the state-of-the-art super-resolution approaches in terms of both PSNR and SSIM performance.https://www.mdpi.com/1424-8220/22/11/4231image super-resolutiondynamic convolution kernelregularized patternmulti-task learningRPB-RDN |
| spellingShingle | Hesen Feng Lihong Ma Jing Tian A Dynamic Convolution Kernel Generation Method Based on Regularized Pattern for Image Super-Resolution Sensors image super-resolution dynamic convolution kernel regularized pattern multi-task learning RPB-RDN |
| title | A Dynamic Convolution Kernel Generation Method Based on Regularized Pattern for Image Super-Resolution |
| title_full | A Dynamic Convolution Kernel Generation Method Based on Regularized Pattern for Image Super-Resolution |
| title_fullStr | A Dynamic Convolution Kernel Generation Method Based on Regularized Pattern for Image Super-Resolution |
| title_full_unstemmed | A Dynamic Convolution Kernel Generation Method Based on Regularized Pattern for Image Super-Resolution |
| title_short | A Dynamic Convolution Kernel Generation Method Based on Regularized Pattern for Image Super-Resolution |
| title_sort | dynamic convolution kernel generation method based on regularized pattern for image super resolution |
| topic | image super-resolution dynamic convolution kernel regularized pattern multi-task learning RPB-RDN |
| url | https://www.mdpi.com/1424-8220/22/11/4231 |
| work_keys_str_mv | AT hesenfeng adynamicconvolutionkernelgenerationmethodbasedonregularizedpatternforimagesuperresolution AT lihongma adynamicconvolutionkernelgenerationmethodbasedonregularizedpatternforimagesuperresolution AT jingtian adynamicconvolutionkernelgenerationmethodbasedonregularizedpatternforimagesuperresolution AT hesenfeng dynamicconvolutionkernelgenerationmethodbasedonregularizedpatternforimagesuperresolution AT lihongma dynamicconvolutionkernelgenerationmethodbasedonregularizedpatternforimagesuperresolution AT jingtian dynamicconvolutionkernelgenerationmethodbasedonregularizedpatternforimagesuperresolution |