PrefAce: face-centric pretraining with self-structure aware distillation
Video-based facial analysis is important for autonomous agents to understand human expressions and sentiments. However, limited labeled data is available to learn effective facial representations. This paper proposes a novel self-supervised face-centric pretraining framework, called PrefAce, which l...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/175280 |
| _version_ | 1811687914199842816 |
|---|---|
| author | Hu, Siyuan |
| author2 | Ong Yew Soon |
| author_facet | Ong Yew Soon Hu, Siyuan |
| author_sort | Hu, Siyuan |
| collection | NTU |
| description | Video-based facial analysis is important for autonomous agents to understand human expressions and sentiments. However, limited labeled data is available to learn effective facial representations. This paper proposes a novel self-supervised face-centric pretraining framework, called PrefAce, which learns transferable video facial representation without labels. The self-supervised learning is performed with an effective landmark-guided global-local tube distillation. Meanwhile, a novel instance-wise update FaceFeat Cache is built to enforce more discriminative and diverse representations for downstream tasks. Extensive experiments demonstrate that the proposed framework learns universal instance-aware facial representations with fine-grained landmark details from videos. The point is that it can transfer across various facial analysis tasks, e.g., Facial Attribute Recognition (FAR), Facial Expression Recognition (FER), DeepFake Detection (DFD), and Lip Synchronization (LS). Our framework also outperforms the state-of-the-art on various downstream tasks, even in low data regimes. |
| first_indexed | 2024-10-01T05:23:53Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/175280 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:23:53Z |
| publishDate | 2024 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1752802024-04-26T15:43:55Z PrefAce: face-centric pretraining with self-structure aware distillation Hu, Siyuan Ong Yew Soon Wen Bihan School of Computer Science and Engineering ASYSOng@ntu.edu.sg, bihan.wen@ntu.edu.sg Computer and Information Science Artificial intelligence Computer vision Video-based facial analysis is important for autonomous agents to understand human expressions and sentiments. However, limited labeled data is available to learn effective facial representations. This paper proposes a novel self-supervised face-centric pretraining framework, called PrefAce, which learns transferable video facial representation without labels. The self-supervised learning is performed with an effective landmark-guided global-local tube distillation. Meanwhile, a novel instance-wise update FaceFeat Cache is built to enforce more discriminative and diverse representations for downstream tasks. Extensive experiments demonstrate that the proposed framework learns universal instance-aware facial representations with fine-grained landmark details from videos. The point is that it can transfer across various facial analysis tasks, e.g., Facial Attribute Recognition (FAR), Facial Expression Recognition (FER), DeepFake Detection (DFD), and Lip Synchronization (LS). Our framework also outperforms the state-of-the-art on various downstream tasks, even in low data regimes. Bachelor's degree 2024-04-23T02:03:03Z 2024-04-23T02:03:03Z 2024 Final Year Project (FYP) Hu, S. (2024). PrefAce: face-centric pretraining with self-structure aware distillation. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/175280 https://hdl.handle.net/10356/175280 en application/pdf Nanyang Technological University |
| spellingShingle | Computer and Information Science Artificial intelligence Computer vision Hu, Siyuan PrefAce: face-centric pretraining with self-structure aware distillation |
| title | PrefAce: face-centric pretraining with self-structure aware distillation |
| title_full | PrefAce: face-centric pretraining with self-structure aware distillation |
| title_fullStr | PrefAce: face-centric pretraining with self-structure aware distillation |
| title_full_unstemmed | PrefAce: face-centric pretraining with self-structure aware distillation |
| title_short | PrefAce: face-centric pretraining with self-structure aware distillation |
| title_sort | preface face centric pretraining with self structure aware distillation |
| topic | Computer and Information Science Artificial intelligence Computer vision |
| url | https://hdl.handle.net/10356/175280 |
| work_keys_str_mv | AT husiyuan prefacefacecentricpretrainingwithselfstructureawaredistillation |