Toward extreme face super-resolution in the wild: A self-supervised learning approach
Extreme face super-resolution (FSR), that is, improving the resolution of face images by an extreme scaling factor (often greater than ×8) has remained underexplored in the literature of low-level vision. Extreme FSR in the wild must address the challenges of both unpaired training data and unknown...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-11-01
|
| Series: | Frontiers in Computer Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fcomp.2022.1037435/full |
| _version_ | 1811306506575937536 |
|---|---|
| author | Ahmed Cheikh Sidiya Xin Li |
| author_facet | Ahmed Cheikh Sidiya Xin Li |
| author_sort | Ahmed Cheikh Sidiya |
| collection | DOAJ |
| description | Extreme face super-resolution (FSR), that is, improving the resolution of face images by an extreme scaling factor (often greater than ×8) has remained underexplored in the literature of low-level vision. Extreme FSR in the wild must address the challenges of both unpaired training data and unknown degradation factors. Inspired by the latest advances in image super-resolution (SR) and self-supervised learning (SSL), we propose a novel two-step approach to FSR by introducing a mid-resolution (MR) image as the stepping stone. In the first step, we leverage ideas from SSL-based SR reconstruction of medical images (e.g., MRI and ultrasound) to modeling the realistic degradation process of face images in the real world; in the second step, we extract the latent codes from MR images and interpolate them in a self-supervised manner to facilitate artifact-suppressed image reconstruction. Our two-step extreme FSR can be interpreted as the combination of existing self-supervised CycleGAN (step 1) and StyleGAN (step 2) that overcomes the barrier of critical resolution in face recognition. Extensive experimental results have shown that our two-step approach can significantly outperform existing state-of-the-art FSR techniques, including FSRGAN, Bulat's method, and PULSE, especially for large scaling factors such as 64. |
| first_indexed | 2024-04-13T08:46:52Z |
| format | Article |
| id | doaj.art-a7f4b283586342c79027c8f46308defe |
| institution | Directory Open Access Journal |
| issn | 2624-9898 |
| language | English |
| last_indexed | 2024-04-13T08:46:52Z |
| publishDate | 2022-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Computer Science |
| spelling | doaj.art-a7f4b283586342c79027c8f46308defe2022-12-22T02:53:38ZengFrontiers Media S.A.Frontiers in Computer Science2624-98982022-11-01410.3389/fcomp.2022.10374351037435Toward extreme face super-resolution in the wild: A self-supervised learning approachAhmed Cheikh SidiyaXin LiExtreme face super-resolution (FSR), that is, improving the resolution of face images by an extreme scaling factor (often greater than ×8) has remained underexplored in the literature of low-level vision. Extreme FSR in the wild must address the challenges of both unpaired training data and unknown degradation factors. Inspired by the latest advances in image super-resolution (SR) and self-supervised learning (SSL), we propose a novel two-step approach to FSR by introducing a mid-resolution (MR) image as the stepping stone. In the first step, we leverage ideas from SSL-based SR reconstruction of medical images (e.g., MRI and ultrasound) to modeling the realistic degradation process of face images in the real world; in the second step, we extract the latent codes from MR images and interpolate them in a self-supervised manner to facilitate artifact-suppressed image reconstruction. Our two-step extreme FSR can be interpreted as the combination of existing self-supervised CycleGAN (step 1) and StyleGAN (step 2) that overcomes the barrier of critical resolution in face recognition. Extensive experimental results have shown that our two-step approach can significantly outperform existing state-of-the-art FSR techniques, including FSRGAN, Bulat's method, and PULSE, especially for large scaling factors such as 64.https://www.frontiersin.org/articles/10.3389/fcomp.2022.1037435/fullextreme face super-resolutionself-supervised learningdegradation learninglatent space interpolationface in the wild |
| spellingShingle | Ahmed Cheikh Sidiya Xin Li Toward extreme face super-resolution in the wild: A self-supervised learning approach Frontiers in Computer Science extreme face super-resolution self-supervised learning degradation learning latent space interpolation face in the wild |
| title | Toward extreme face super-resolution in the wild: A self-supervised learning approach |
| title_full | Toward extreme face super-resolution in the wild: A self-supervised learning approach |
| title_fullStr | Toward extreme face super-resolution in the wild: A self-supervised learning approach |
| title_full_unstemmed | Toward extreme face super-resolution in the wild: A self-supervised learning approach |
| title_short | Toward extreme face super-resolution in the wild: A self-supervised learning approach |
| title_sort | toward extreme face super resolution in the wild a self supervised learning approach |
| topic | extreme face super-resolution self-supervised learning degradation learning latent space interpolation face in the wild |
| url | https://www.frontiersin.org/articles/10.3389/fcomp.2022.1037435/full |
| work_keys_str_mv | AT ahmedcheikhsidiya towardextremefacesuperresolutioninthewildaselfsupervisedlearningapproach AT xinli towardextremefacesuperresolutioninthewildaselfsupervisedlearningapproach |