Few-shot forgery detection via guided adversarial interpolation
The increase in face manipulation models has led to a critical issue in society - the synthesis of realistic visual media. With the emergence of new forgery approaches at an unprecedented rate, existing forgery detection methods suffer from significant performance drops when applied to unseen nov...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/171185 |
| _version_ | 1811689908272627712 |
|---|---|
| author | Qiu, Haonan Chen, Siyu Gan, Bei Wang, Kun Shi, Huafeng Shao, Jing Liu, Ziwei |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Qiu, Haonan Chen, Siyu Gan, Bei Wang, Kun Shi, Huafeng Shao, Jing Liu, Ziwei |
| author_sort | Qiu, Haonan |
| collection | NTU |
| description | The increase in face manipulation models has led to a critical issue in
society - the synthesis of realistic visual media. With the emergence of new
forgery approaches at an unprecedented rate, existing forgery detection methods
suffer from significant performance drops when applied to unseen novel forgery
approaches. In this work, we address the few-shot forgery detection problem by
1) designing a comprehensive benchmark based on coverage analysis among various
forgery approaches, and 2) proposing Guided Adversarial Interpolation (GAI).
Our key insight is that there exist transferable distribution characteristics
between majority and minority forgery classes1. Specifically, we enhance the
discriminative ability against novel forgery approaches via adversarially
interpolating the forgery artifacts of the minority samples to the majority
samples under the guidance of a teacher network. Unlike the standard
re-balancing method which usually results in over-fitting to minority classes,
our method simultaneously takes account of the diversity of majority
information as well as the significance of minority information. Extensive
experiments demonstrate that our GAI achieves state-of-the-art performances on
the established few-shot forgery detection benchmark. Notably, our method is
also validated to be robust to choices of majority and minority forgery
approaches. |
| first_indexed | 2024-10-01T05:55:34Z |
| format | Journal Article |
| id | ntu-10356/171185 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:55:34Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | ntu-10356/1711852023-10-17T01:39:45Z Few-shot forgery detection via guided adversarial interpolation Qiu, Haonan Chen, Siyu Gan, Bei Wang, Kun Shi, Huafeng Shao, Jing Liu, Ziwei School of Computer Science and Engineering S-Lab Engineering::Computer science and engineering Forgery Detection DeepFake The increase in face manipulation models has led to a critical issue in society - the synthesis of realistic visual media. With the emergence of new forgery approaches at an unprecedented rate, existing forgery detection methods suffer from significant performance drops when applied to unseen novel forgery approaches. In this work, we address the few-shot forgery detection problem by 1) designing a comprehensive benchmark based on coverage analysis among various forgery approaches, and 2) proposing Guided Adversarial Interpolation (GAI). Our key insight is that there exist transferable distribution characteristics between majority and minority forgery classes1. Specifically, we enhance the discriminative ability against novel forgery approaches via adversarially interpolating the forgery artifacts of the minority samples to the majority samples under the guidance of a teacher network. Unlike the standard re-balancing method which usually results in over-fitting to minority classes, our method simultaneously takes account of the diversity of majority information as well as the significance of minority information. Extensive experiments demonstrate that our GAI achieves state-of-the-art performances on the established few-shot forgery detection benchmark. Notably, our method is also validated to be robust to choices of majority and minority forgery approaches. Agency for Science, Technology and Research (A*STAR) Ministry of Education (MOE) National Research Foundation (NRF) This research/project is supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award No: AISG2-PhD-2022-01-035T), NTU NAP, Singapore, MOE AcRF Tier 1 (2021-T1-001-088), and under the RIE2020 Industry Alignment Fund – Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s). 2023-10-17T01:39:45Z 2023-10-17T01:39:45Z 2023 Journal Article Qiu, H., Chen, S., Gan, B., Wang, K., Shi, H., Shao, J. & Liu, Z. (2023). Few-shot forgery detection via guided adversarial interpolation. Pattern Recognition, 144, 109863-. https://dx.doi.org/10.1016/j.patcog.2023.109863 0031-3203 https://hdl.handle.net/10356/171185 10.1016/j.patcog.2023.109863 2-s2.0-85167995613 144 109863 en AISG2-PhD-2022-01-035T NTU NAP 2021-T1-001-088 Pattern Recognition © 2023 Elsevier Ltd. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering Forgery Detection DeepFake Qiu, Haonan Chen, Siyu Gan, Bei Wang, Kun Shi, Huafeng Shao, Jing Liu, Ziwei Few-shot forgery detection via guided adversarial interpolation |
| title | Few-shot forgery detection via guided adversarial interpolation |
| title_full | Few-shot forgery detection via guided adversarial interpolation |
| title_fullStr | Few-shot forgery detection via guided adversarial interpolation |
| title_full_unstemmed | Few-shot forgery detection via guided adversarial interpolation |
| title_short | Few-shot forgery detection via guided adversarial interpolation |
| title_sort | few shot forgery detection via guided adversarial interpolation |
| topic | Engineering::Computer science and engineering Forgery Detection DeepFake |
| url | https://hdl.handle.net/10356/171185 |
| work_keys_str_mv | AT qiuhaonan fewshotforgerydetectionviaguidedadversarialinterpolation AT chensiyu fewshotforgerydetectionviaguidedadversarialinterpolation AT ganbei fewshotforgerydetectionviaguidedadversarialinterpolation AT wangkun fewshotforgerydetectionviaguidedadversarialinterpolation AT shihuafeng fewshotforgerydetectionviaguidedadversarialinterpolation AT shaojing fewshotforgerydetectionviaguidedadversarialinterpolation AT liuziwei fewshotforgerydetectionviaguidedadversarialinterpolation |