Exploiting the image prior in CLIP for super-resolution
Super-resolution (SR) is a fundamental task in computer vision aimed at enhancing the resolution and quality of low-resolution images. However, a persistent challenge arises from the inherent ambiguity where a single low-resolution image may correspond to mul- tiple high-resolution images. Additiona...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/175133 |
| _version_ | 1826115365273337856 |
|---|---|
| author | Chen, Xingyu |
| author2 | Chen Change Loy |
| author_facet | Chen Change Loy Chen, Xingyu |
| author_sort | Chen, Xingyu |
| collection | NTU |
| description | Super-resolution (SR) is a fundamental task in computer vision aimed at enhancing the resolution and quality of low-resolution images. However, a persistent challenge arises from the inherent ambiguity where a single low-resolution image may correspond to mul- tiple high-resolution images. Additional priors are essential to address such problem, especially when the degradation is complex. Recent emergence of large vision-language model such as CLIP provides potential to enhance SR generation by providing extra con- textual information from the image. Hence, in this project, we investigate the efficacy of integrating CLIP priors into image super-resolution. Through a series of experiments, we explore both blind and non-blind SR problems, evaluating the impact of CLIP priors on model performance. Additionally, we analyze the limitations and challenges associated with CLIP integration, particularly in handling low-resolution and incomplete images. Our findings demonstrate that while CLIP priors hold promise in enhancing SR results, careful fine-tuning is required to optimize their utilization for image generation tasks. |
| first_indexed | 2024-10-01T03:54:16Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/175133 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:54:16Z |
| publishDate | 2024 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1751332024-04-26T15:40:52Z Exploiting the image prior in CLIP for super-resolution Chen, Xingyu Chen Change Loy School of Computer Science and Engineering ccloy@ntu.edu.sg Computer and Information Science Super resolution Computer vision CLIP Deep learning Super-resolution (SR) is a fundamental task in computer vision aimed at enhancing the resolution and quality of low-resolution images. However, a persistent challenge arises from the inherent ambiguity where a single low-resolution image may correspond to mul- tiple high-resolution images. Additional priors are essential to address such problem, especially when the degradation is complex. Recent emergence of large vision-language model such as CLIP provides potential to enhance SR generation by providing extra con- textual information from the image. Hence, in this project, we investigate the efficacy of integrating CLIP priors into image super-resolution. Through a series of experiments, we explore both blind and non-blind SR problems, evaluating the impact of CLIP priors on model performance. Additionally, we analyze the limitations and challenges associated with CLIP integration, particularly in handling low-resolution and incomplete images. Our findings demonstrate that while CLIP priors hold promise in enhancing SR results, careful fine-tuning is required to optimize their utilization for image generation tasks. Bachelor's degree 2024-04-22T02:49:53Z 2024-04-22T02:49:53Z 2024 Final Year Project (FYP) Chen, X. (2024). Exploiting the image prior in CLIP for super-resolution. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/175133 https://hdl.handle.net/10356/175133 en SCSE23-0477 application/pdf Nanyang Technological University |
| spellingShingle | Computer and Information Science Super resolution Computer vision CLIP Deep learning Chen, Xingyu Exploiting the image prior in CLIP for super-resolution |
| title | Exploiting the image prior in CLIP for super-resolution |
| title_full | Exploiting the image prior in CLIP for super-resolution |
| title_fullStr | Exploiting the image prior in CLIP for super-resolution |
| title_full_unstemmed | Exploiting the image prior in CLIP for super-resolution |
| title_short | Exploiting the image prior in CLIP for super-resolution |
| title_sort | exploiting the image prior in clip for super resolution |
| topic | Computer and Information Science Super resolution Computer vision CLIP Deep learning |
| url | https://hdl.handle.net/10356/175133 |
| work_keys_str_mv | AT chenxingyu exploitingtheimagepriorinclipforsuperresolution |