Detecting Compressed Deepfake Images Using Two-Branch Convolutional Networks with Similarity and Classifier
As a popular technique for swapping faces with someone else’s in images or videos through deep neural networks, deepfake causes a serious threat to the security of multimedia content today. However, because counterfeit images are usually compressed when propagating over the Internet, and because the...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Symmetry |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-8994/14/12/2691 |
| _version_ | 1797455179996463104 |
|---|---|
| author | Ping Chen Ming Xu Xiaodong Wang |
| author_facet | Ping Chen Ming Xu Xiaodong Wang |
| author_sort | Ping Chen |
| collection | DOAJ |
| description | As a popular technique for swapping faces with someone else’s in images or videos through deep neural networks, deepfake causes a serious threat to the security of multimedia content today. However, because counterfeit images are usually compressed when propagating over the Internet, and because the compression factor used is unknown, most of the existing deepfake detection models have poor robustness for the detection of compressed images with unknown compression factors. To solve this problem, we notice that an image has a high similarity with its compressed image based on symmetry, and this similarity is not easily affected by the compression factor, so this similarity feature can be used as an important clue for compressed deepfake detection. A TCNSC (Two-branch Convolutional Networks with Similarity and Classifier) method that combines compression factor independence is proposed in this paper. The TCNSC method learns two feature representations from the deepfake image, i.e., similarity of the image and its compressed counterpart and authenticity of the deepfake image. A joint training strategy is then utilized for feature extraction, in which the similarity characteristics are obtained by similarity learning while obtaining authenticity characteristics, so the proposed TCNSC model is trained for robust feature learning. Experimental results on the FaceForensics++ (FF++) dataset show that the proposed method significantly outperforms all competing methods under three compression settings of high-quality (HQ), medium-quality (MQ), and low-quality (LQ). For the LQ, MQ, and HQ settings, TCNSC achieves 91.8%, 93.4%, and 95.3% in accuracy, and outperforms the state-of-art method (Xception-RAW) by 16.9%, 10.1%, and 4.1%, respectively. |
| first_indexed | 2024-03-09T15:47:48Z |
| format | Article |
| id | doaj.art-891070f724e941b2bbabd5ed9b881926 |
| institution | Directory Open Access Journal |
| issn | 2073-8994 |
| language | English |
| last_indexed | 2024-03-09T15:47:48Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Symmetry |
| spelling | doaj.art-891070f724e941b2bbabd5ed9b8819262023-11-24T18:21:30ZengMDPI AGSymmetry2073-89942022-12-011412269110.3390/sym14122691Detecting Compressed Deepfake Images Using Two-Branch Convolutional Networks with Similarity and ClassifierPing Chen0Ming Xu1Xiaodong Wang2School of Cyberspace, Hangzhou Dianzi University, Hangzhou 310018, ChinaSchool of Cyberspace, Hangzhou Dianzi University, Hangzhou 310018, ChinaEducation Technology Center of Zhejiang Province, Hangzhou 310030, ChinaAs a popular technique for swapping faces with someone else’s in images or videos through deep neural networks, deepfake causes a serious threat to the security of multimedia content today. However, because counterfeit images are usually compressed when propagating over the Internet, and because the compression factor used is unknown, most of the existing deepfake detection models have poor robustness for the detection of compressed images with unknown compression factors. To solve this problem, we notice that an image has a high similarity with its compressed image based on symmetry, and this similarity is not easily affected by the compression factor, so this similarity feature can be used as an important clue for compressed deepfake detection. A TCNSC (Two-branch Convolutional Networks with Similarity and Classifier) method that combines compression factor independence is proposed in this paper. The TCNSC method learns two feature representations from the deepfake image, i.e., similarity of the image and its compressed counterpart and authenticity of the deepfake image. A joint training strategy is then utilized for feature extraction, in which the similarity characteristics are obtained by similarity learning while obtaining authenticity characteristics, so the proposed TCNSC model is trained for robust feature learning. Experimental results on the FaceForensics++ (FF++) dataset show that the proposed method significantly outperforms all competing methods under three compression settings of high-quality (HQ), medium-quality (MQ), and low-quality (LQ). For the LQ, MQ, and HQ settings, TCNSC achieves 91.8%, 93.4%, and 95.3% in accuracy, and outperforms the state-of-art method (Xception-RAW) by 16.9%, 10.1%, and 4.1%, respectively.https://www.mdpi.com/2073-8994/14/12/2691compressed deepfake detectionsimilarity learningforgery detectionmultimedia forensics |
| spellingShingle | Ping Chen Ming Xu Xiaodong Wang Detecting Compressed Deepfake Images Using Two-Branch Convolutional Networks with Similarity and Classifier Symmetry compressed deepfake detection similarity learning forgery detection multimedia forensics |
| title | Detecting Compressed Deepfake Images Using Two-Branch Convolutional Networks with Similarity and Classifier |
| title_full | Detecting Compressed Deepfake Images Using Two-Branch Convolutional Networks with Similarity and Classifier |
| title_fullStr | Detecting Compressed Deepfake Images Using Two-Branch Convolutional Networks with Similarity and Classifier |
| title_full_unstemmed | Detecting Compressed Deepfake Images Using Two-Branch Convolutional Networks with Similarity and Classifier |
| title_short | Detecting Compressed Deepfake Images Using Two-Branch Convolutional Networks with Similarity and Classifier |
| title_sort | detecting compressed deepfake images using two branch convolutional networks with similarity and classifier |
| topic | compressed deepfake detection similarity learning forgery detection multimedia forensics |
| url | https://www.mdpi.com/2073-8994/14/12/2691 |
| work_keys_str_mv | AT pingchen detectingcompresseddeepfakeimagesusingtwobranchconvolutionalnetworkswithsimilarityandclassifier AT mingxu detectingcompresseddeepfakeimagesusingtwobranchconvolutionalnetworkswithsimilarityandclassifier AT xiaodongwang detectingcompresseddeepfakeimagesusingtwobranchconvolutionalnetworkswithsimilarityandclassifier |