In-the-wild image quality assessment with diffusion priors
Blind image quality assessment (IQA) in the wild, which assesses the quality of images with complex authentic distortions and no reference images, presents significant challenges. Given the difficulty in collecting large-scale training data, leveraging limited data to develop a model with strong gen...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Master by Coursework |
| Language: | English |
| Published: |
Nanyang Technological University
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/181882 |
| _version_ | 1826130302464950272 |
|---|---|
| author | Fu, Honghao |
| author2 | Wen Bihan |
| author_facet | Wen Bihan Fu, Honghao |
| author_sort | Fu, Honghao |
| collection | NTU |
| description | Blind image quality assessment (IQA) in the wild, which assesses the quality of images with complex authentic distortions and no reference images, presents significant challenges. Given the difficulty in collecting large-scale training data, leveraging limited data to develop a model with strong generalization remains an open problem. Motivated by the robust image perception capabilities of pretrained text-to image (T2I) diffusion models, we propose a novel IQA method, diffusion priors-based IQA (DP-IQA), to utilize the T2I model’s prior for improved performance and generalization ability. Specifically, we utilize pre-trained Stable Diffusion as the backbone, extracting multi-level features from the denoising U-Net guided by prompt embeddings through a tunable text adapter. Simultaneously, an image adapter compensates for information loss introduced by the lossy pre-trained encoder. Unlike T2I models that require full image distribution modeling, our approach targets image quality assessment, which inherently requires fewer parameters. To improve applicability, we distill the knowledge into a lightweight CNN-based student model, significantly reducing parameters while maintaining or even enhancing generalization performance. Experimental results demonstrate that DP-IQA achieves state-of-the-art performance on various in-the-wild datasets, highlighting the superior generalization capability of T2I priors in blind IQA tasks. To our knowledge, DP-IQA is the first method to apply pre-trained diffusion priors in blind IQA. |
| first_indexed | 2025-03-09T15:19:27Z |
| format | Thesis-Master by Coursework |
| id | ntu-10356/181882 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2025-03-09T15:19:27Z |
| publishDate | 2024 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1818822024-12-27T15:46:13Z In-the-wild image quality assessment with diffusion priors Fu, Honghao Wen Bihan School of Electrical and Electronic Engineering Rapid-Rich Object Search (ROSE) Lab bihan.wen@ntu.edu.sg Computer and Information Science Blind IQA Diffusion prior Text-to-image model Knowledge distillation Blind image quality assessment (IQA) in the wild, which assesses the quality of images with complex authentic distortions and no reference images, presents significant challenges. Given the difficulty in collecting large-scale training data, leveraging limited data to develop a model with strong generalization remains an open problem. Motivated by the robust image perception capabilities of pretrained text-to image (T2I) diffusion models, we propose a novel IQA method, diffusion priors-based IQA (DP-IQA), to utilize the T2I model’s prior for improved performance and generalization ability. Specifically, we utilize pre-trained Stable Diffusion as the backbone, extracting multi-level features from the denoising U-Net guided by prompt embeddings through a tunable text adapter. Simultaneously, an image adapter compensates for information loss introduced by the lossy pre-trained encoder. Unlike T2I models that require full image distribution modeling, our approach targets image quality assessment, which inherently requires fewer parameters. To improve applicability, we distill the knowledge into a lightweight CNN-based student model, significantly reducing parameters while maintaining or even enhancing generalization performance. Experimental results demonstrate that DP-IQA achieves state-of-the-art performance on various in-the-wild datasets, highlighting the superior generalization capability of T2I priors in blind IQA tasks. To our knowledge, DP-IQA is the first method to apply pre-trained diffusion priors in blind IQA. Master's degree 2024-12-27T13:31:16Z 2024-12-27T13:31:16Z 2024 Thesis-Master by Coursework Fu, H. (2024). In-the-wild image quality assessment with diffusion priors. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/181882 https://hdl.handle.net/10356/181882 en application/pdf Nanyang Technological University |
| spellingShingle | Computer and Information Science Blind IQA Diffusion prior Text-to-image model Knowledge distillation Fu, Honghao In-the-wild image quality assessment with diffusion priors |
| title | In-the-wild image quality assessment with diffusion priors |
| title_full | In-the-wild image quality assessment with diffusion priors |
| title_fullStr | In-the-wild image quality assessment with diffusion priors |
| title_full_unstemmed | In-the-wild image quality assessment with diffusion priors |
| title_short | In-the-wild image quality assessment with diffusion priors |
| title_sort | in the wild image quality assessment with diffusion priors |
| topic | Computer and Information Science Blind IQA Diffusion prior Text-to-image model Knowledge distillation |
| url | https://hdl.handle.net/10356/181882 |
| work_keys_str_mv | AT fuhonghao inthewildimagequalityassessmentwithdiffusionpriors |