Watermarking for combating deepfakes
Over the recent years, great concerns have been aroused around the topic of Deefake due to its amazing ability in making a forgery image look like a genuine one. Many approaches have been developed to alleviate such risks. Among these, one noticeable track is to apply the model’s adversarial nois...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/165932 |
| _version_ | 1811683765456470016 |
|---|---|
| author | Li, Rui |
| author2 | Lin Weisi |
| author_facet | Lin Weisi Li, Rui |
| author_sort | Li, Rui |
| collection | NTU |
| description | Over the recent years, great concerns have been aroused around the topic of Deefake
due to its amazing ability in making a forgery image look like a genuine one. Many
approaches have been developed to alleviate such risks. Among these, one noticeable
track is to apply the model’s adversarial noise as a watermark to the image so that
when the image is modified, it would be drastically distorted to the extent that the
person’s facial features are no longer recognizable. Recent works have successfully
developed a cross-model universal attack method that can produce a watermark that
can protect multiple images against multiple models, breaking the previous constraint
of watermarks being image-model-specific. However, to ensure the desired level of
distortion, the adversarial noise threshold is set to relatively high, which makes the
watermark ultimately visible on human faces, impairing the image quality and aesthetic.
To mitigate this issue, we bring the idea of just noticeable difference (JND) into the
cross-model universal attack method, intending to produce an image quality preserved
universal watermark, while still maintaining the original protection performance. To
achieve this, we have made several attempts. First, we replace the threshold clamp at
each attacking step with the JND clamp. Second, we introduce a face parsing model
to gain finer control over the JND values. Specifically, we use the face parsing model
to segment portrait images into different parts and add scaling factors respectively for
each part to scale the JND values. Through this, we are able to achieve good visual
quality and at the same time, maintain good protection performance. Experiments
are conducted to show that the watermark produced from the new JND cross-model
universal watermark outperforms the previous one both in visual quality and protection
performance. |
| first_indexed | 2024-10-01T04:17:56Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/165932 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T04:17:56Z |
| publishDate | 2023 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1659322023-04-21T15:37:01Z Watermarking for combating deepfakes Li, Rui Lin Weisi School of Computer Science and Engineering WSLin@ntu.edu.sg Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Over the recent years, great concerns have been aroused around the topic of Deefake due to its amazing ability in making a forgery image look like a genuine one. Many approaches have been developed to alleviate such risks. Among these, one noticeable track is to apply the model’s adversarial noise as a watermark to the image so that when the image is modified, it would be drastically distorted to the extent that the person’s facial features are no longer recognizable. Recent works have successfully developed a cross-model universal attack method that can produce a watermark that can protect multiple images against multiple models, breaking the previous constraint of watermarks being image-model-specific. However, to ensure the desired level of distortion, the adversarial noise threshold is set to relatively high, which makes the watermark ultimately visible on human faces, impairing the image quality and aesthetic. To mitigate this issue, we bring the idea of just noticeable difference (JND) into the cross-model universal attack method, intending to produce an image quality preserved universal watermark, while still maintaining the original protection performance. To achieve this, we have made several attempts. First, we replace the threshold clamp at each attacking step with the JND clamp. Second, we introduce a face parsing model to gain finer control over the JND values. Specifically, we use the face parsing model to segment portrait images into different parts and add scaling factors respectively for each part to scale the JND values. Through this, we are able to achieve good visual quality and at the same time, maintain good protection performance. Experiments are conducted to show that the watermark produced from the new JND cross-model universal watermark outperforms the previous one both in visual quality and protection performance. Bachelor of Engineering (Computer Science) 2023-04-17T02:23:50Z 2023-04-17T02:23:50Z 2023 Final Year Project (FYP) Li, R. (2023). Watermarking for combating deepfakes. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/165932 https://hdl.handle.net/10356/165932 en SCSE22001 application/pdf Nanyang Technological University |
| spellingShingle | Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision Li, Rui Watermarking for combating deepfakes |
| title | Watermarking for combating deepfakes |
| title_full | Watermarking for combating deepfakes |
| title_fullStr | Watermarking for combating deepfakes |
| title_full_unstemmed | Watermarking for combating deepfakes |
| title_short | Watermarking for combating deepfakes |
| title_sort | watermarking for combating deepfakes |
| topic | Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision |
| url | https://hdl.handle.net/10356/165932 |
| work_keys_str_mv | AT lirui watermarkingforcombatingdeepfakes |