A novel model compression method based on joint distillation for deepfake video detection
In recent years, deepfake videos have been abused to create fake news, which threaten the integrity of digital videos. Although existing detection methods leveraged cumbersome neural networks to achieve promising detection performance, they cannot be deployed in resource-constrained scenarios. To ov...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-10-01
|
| Series: | Journal of King Saud University: Computer and Information Sciences |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1319157823003464 |
| _version_ | 1797629735854931968 |
|---|---|
| author | Xiong Xu Shuai Tang Mingcheng Zhu Peisong He Sirui Li Yun Cao |
| author_facet | Xiong Xu Shuai Tang Mingcheng Zhu Peisong He Sirui Li Yun Cao |
| author_sort | Xiong Xu |
| collection | DOAJ |
| description | In recent years, deepfake videos have been abused to create fake news, which threaten the integrity of digital videos. Although existing detection methods leveraged cumbersome neural networks to achieve promising detection performance, they cannot be deployed in resource-constrained scenarios. To overcome this limitation, we propose a novel model compression framework based on joint distillation for deepfake detection, which includes a pre-training stage and a knowledge transfer stage. In the pre-training stage, a teacher network is trained with sufficient labeled samples. Then, in the knowledge transfer stage, a lightweight student network is constructed by considering dimension alignment. To transfer forensics knowledge comprehensively, a joint distillation loss is designed, including cross-entropy loss, knowledge distillation loss, and gradient-guided feature distillation loss. For feature distillation, feature maps from both shallow and deep layers are utilized to calculate channel-wise mean square error weighted by gradient information for transferring knowledge of forensics features adaptively. Besides, a decayed teaching strategy is constructed to adjust the importance of feature distillation, which aims at mitigating the risk of negative transfer. Extensive experiments show that student networks obtained by our model compression method can achieve competitive detection performance and outstanding efficiency by distinctly reducing computational costs compared against state-of-the-art methods. |
| first_indexed | 2024-03-11T10:58:11Z |
| format | Article |
| id | doaj.art-5eadbfe5428444c6a40e317de1e812ae |
| institution | Directory Open Access Journal |
| issn | 1319-1578 |
| language | English |
| last_indexed | 2024-03-11T10:58:11Z |
| publishDate | 2023-10-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Journal of King Saud University: Computer and Information Sciences |
| spelling | doaj.art-5eadbfe5428444c6a40e317de1e812ae2023-11-13T04:09:06ZengElsevierJournal of King Saud University: Computer and Information Sciences1319-15782023-10-01359101792A novel model compression method based on joint distillation for deepfake video detectionXiong Xu0Shuai Tang1Mingcheng Zhu2Peisong He3Sirui Li4Yun Cao5Southwest China Institute of Electronic Technology, Chengdu 610036, ChinaSchool of Cyber Science and Engineering, Sichuan University, Chengdu 610207, ChinaDepartment of Computing, Imperial College London, London SW72AZ, United KingdomSchool of Cyber Science and Engineering, Sichuan University, Chengdu 610207, China; Corresponding author.Pittsburgh Institute, Sichuan University, Chengdu 610207, ChinaState Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100085, ChinaIn recent years, deepfake videos have been abused to create fake news, which threaten the integrity of digital videos. Although existing detection methods leveraged cumbersome neural networks to achieve promising detection performance, they cannot be deployed in resource-constrained scenarios. To overcome this limitation, we propose a novel model compression framework based on joint distillation for deepfake detection, which includes a pre-training stage and a knowledge transfer stage. In the pre-training stage, a teacher network is trained with sufficient labeled samples. Then, in the knowledge transfer stage, a lightweight student network is constructed by considering dimension alignment. To transfer forensics knowledge comprehensively, a joint distillation loss is designed, including cross-entropy loss, knowledge distillation loss, and gradient-guided feature distillation loss. For feature distillation, feature maps from both shallow and deep layers are utilized to calculate channel-wise mean square error weighted by gradient information for transferring knowledge of forensics features adaptively. Besides, a decayed teaching strategy is constructed to adjust the importance of feature distillation, which aims at mitigating the risk of negative transfer. Extensive experiments show that student networks obtained by our model compression method can achieve competitive detection performance and outstanding efficiency by distinctly reducing computational costs compared against state-of-the-art methods.http://www.sciencedirect.com/science/article/pii/S1319157823003464Video forensicsDeepfake detectionModel compressionJoint distillationGradient-guided feature distillationDecayed teaching strategy |
| spellingShingle | Xiong Xu Shuai Tang Mingcheng Zhu Peisong He Sirui Li Yun Cao A novel model compression method based on joint distillation for deepfake video detection Journal of King Saud University: Computer and Information Sciences Video forensics Deepfake detection Model compression Joint distillation Gradient-guided feature distillation Decayed teaching strategy |
| title | A novel model compression method based on joint distillation for deepfake video detection |
| title_full | A novel model compression method based on joint distillation for deepfake video detection |
| title_fullStr | A novel model compression method based on joint distillation for deepfake video detection |
| title_full_unstemmed | A novel model compression method based on joint distillation for deepfake video detection |
| title_short | A novel model compression method based on joint distillation for deepfake video detection |
| title_sort | novel model compression method based on joint distillation for deepfake video detection |
| topic | Video forensics Deepfake detection Model compression Joint distillation Gradient-guided feature distillation Decayed teaching strategy |
| url | http://www.sciencedirect.com/science/article/pii/S1319157823003464 |
| work_keys_str_mv | AT xiongxu anovelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT shuaitang anovelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT mingchengzhu anovelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT peisonghe anovelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT siruili anovelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT yuncao anovelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT xiongxu novelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT shuaitang novelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT mingchengzhu novelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT peisonghe novelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT siruili novelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection AT yuncao novelmodelcompressionmethodbasedonjointdistillationfordeepfakevideodetection |