Face super-resolution with large pose variation
Recently many face super-resolution (FSR) algorithms have achieved great progress. The current interest lies in utilizing natural image priors from state of the art pre-trained Generative Adversarial Networks (GAN) to improve the restoration quality. With these pre-trained GAN priors, modern FSR w...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/156468 |
| _version_ | 1826112546491334656 |
|---|---|
| author | Lim, Sheng Zhe |
| author2 | Chen Change Loy |
| author_facet | Chen Change Loy Lim, Sheng Zhe |
| author_sort | Lim, Sheng Zhe |
| collection | NTU |
| description | Recently many face super-resolution (FSR) algorithms have achieved great progress.
The current interest lies in utilizing natural image priors from state of the art pre-trained Generative Adversarial Networks (GAN) to improve the restoration quality.
With these pre-trained GAN priors, modern FSR works can recover rich texture and
fine detail super-resolution (SR) face output from low resolution (LR) face input.
However, while most FSR researchers focus on frontal or semi-frontal face super-resolution, the FSR performance on large face pose always be left out. To address
this situation, I performed an analysis on a novel FSR algorithm - Generative Latent
Bank (GLEAN) and found out that the performance of GLEAN face declined when the
LR face input has a large face pose. Furthermore, through fine-tuning GLEAN with
large face pose training data, I discovered GLEAN is able to learn and recover facial
detail and texture of large face pose despite having GAN priors that are mostly trained
on frontal faces. On top of that, I proposed a new cropped face dataset that contains
over 200k+ images for evaluating FSR performance to encourage the community to
focus on a small scale, extreme pose and heavily corrupted old face super-resolution
problems. Finally, I concluded with future improvement directions on the proposed
fine-tuned GLEAN. |
| first_indexed | 2024-10-01T03:08:49Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/156468 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:08:49Z |
| publishDate | 2022 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1564682022-04-17T11:14:42Z Face super-resolution with large pose variation Lim, Sheng Zhe Chen Change Loy School of Computer Science and Engineering ccloy@ntu.edu.sg Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Recently many face super-resolution (FSR) algorithms have achieved great progress. The current interest lies in utilizing natural image priors from state of the art pre-trained Generative Adversarial Networks (GAN) to improve the restoration quality. With these pre-trained GAN priors, modern FSR works can recover rich texture and fine detail super-resolution (SR) face output from low resolution (LR) face input. However, while most FSR researchers focus on frontal or semi-frontal face super-resolution, the FSR performance on large face pose always be left out. To address this situation, I performed an analysis on a novel FSR algorithm - Generative Latent Bank (GLEAN) and found out that the performance of GLEAN face declined when the LR face input has a large face pose. Furthermore, through fine-tuning GLEAN with large face pose training data, I discovered GLEAN is able to learn and recover facial detail and texture of large face pose despite having GAN priors that are mostly trained on frontal faces. On top of that, I proposed a new cropped face dataset that contains over 200k+ images for evaluating FSR performance to encourage the community to focus on a small scale, extreme pose and heavily corrupted old face super-resolution problems. Finally, I concluded with future improvement directions on the proposed fine-tuned GLEAN. Bachelor of Engineering (Computer Science) 2022-04-17T11:14:42Z 2022-04-17T11:14:42Z 2022 Final Year Project (FYP) Lim, S. Z. (2022). Face super-resolution with large pose variation. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/156468 https://hdl.handle.net/10356/156468 en SCSE21-0203 application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Lim, Sheng Zhe Face super-resolution with large pose variation |
| title | Face super-resolution with large pose variation |
| title_full | Face super-resolution with large pose variation |
| title_fullStr | Face super-resolution with large pose variation |
| title_full_unstemmed | Face super-resolution with large pose variation |
| title_short | Face super-resolution with large pose variation |
| title_sort | face super resolution with large pose variation |
| topic | Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision |
| url | https://hdl.handle.net/10356/156468 |
| work_keys_str_mv | AT limshengzhe facesuperresolutionwithlargeposevariation |