Adversarial examples defense method based on multi-dimensional feature maps knowledge distillation
The neural network approach has been commonly used in computer vision tasks.However, adversarial examples are able to make a neural network generate a false prediction.Adversarial training has been shown to be an effective approach to defend against the impact of adversarial examples.Nevertheless, i...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
POSTS&TELECOM PRESS Co., LTD
2022-04-01
|
| Series: | 网络与信息安全学报 |
| Subjects: | |
| Online Access: | http://www.infocomm-journal.com/cjnis/CN/10.11959/j.issn.2096-109x.2022012 |
| _version_ | 1811246714418364416 |
|---|---|
| author | Baolin QIU, Ping YI |
| author_facet | Baolin QIU, Ping YI |
| author_sort | Baolin QIU, Ping YI |
| collection | DOAJ |
| description | The neural network approach has been commonly used in computer vision tasks.However, adversarial examples are able to make a neural network generate a false prediction.Adversarial training has been shown to be an effective approach to defend against the impact of adversarial examples.Nevertheless, it requires high computing power and long training time thus limiting its application scenarios.An adversarial examples defense method based on knowledge distillation was proposed, reusing the defense experience from the large datasets to new classification tasks.During distillation, teacher model has the same structure as student model and the feature map vector was used to transfer experience, and clean samples were used for training.Multi-dimensional feature maps were utilized to enhance the semantic information.Furthermore, an attention mechanism based on feature map was proposed, which boosted the effect of distillation by assigning weights to features according to their importance.Experiments were conducted over cifar100 and cifar10 open-source dataset.And various white-box attack algorithms such as FGSM (fast gradient sign method), PGD (project gradient descent) and C＆amp;W (Carlini-Wagner attack) were applied to test the experimental results.The accuracy of the proposed method on Cifar10 clean samples exceeds that of adversarial training and is close to the accuracy of the model trained on clean samples.Under the PGD attack of L2 distance, the efficiency of the proposed method is close to that of adversarial training, which is significantly higher than that of normal training.Moreover, the proposed method is a light-weight adversarial defense method with low learning cost.The computing power requirement is far less than that of adversarial training even if optimization schemes such as attention mechanism and multi-dimensional feature map are added.Knowledge distillation can learn the decision-making experience of normal samples and extract robust features as a neural network learning scheme.It uses a small amount of data to generate accurate and robust models, improves generalization, and reduces the cost of adversarial training. |
| first_indexed | 2024-04-12T14:57:24Z |
| format | Article |
| id | doaj.art-c2ee2fc052f14785a958402f6ebded2a |
| institution | Directory Open Access Journal |
| issn | 2096-109X |
| language | English |
| last_indexed | 2024-04-12T14:57:24Z |
| publishDate | 2022-04-01 |
| publisher | POSTS&TELECOM PRESS Co., LTD |
| record_format | Article |
| series | 网络与信息安全学报 |
| spelling | doaj.art-c2ee2fc052f14785a958402f6ebded2a2022-12-22T03:28:11ZengPOSTS&TELECOM PRESS Co., LTD网络与信息安全学报2096-109X2022-04-0182889910.11959/j.issn.2096-109x.2022012Adversarial examples defense method based on multi-dimensional feature maps knowledge distillationBaolin QIU, Ping YI 0School of Cyber Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, ChinaThe neural network approach has been commonly used in computer vision tasks.However, adversarial examples are able to make a neural network generate a false prediction.Adversarial training has been shown to be an effective approach to defend against the impact of adversarial examples.Nevertheless, it requires high computing power and long training time thus limiting its application scenarios.An adversarial examples defense method based on knowledge distillation was proposed, reusing the defense experience from the large datasets to new classification tasks.During distillation, teacher model has the same structure as student model and the feature map vector was used to transfer experience, and clean samples were used for training.Multi-dimensional feature maps were utilized to enhance the semantic information.Furthermore, an attention mechanism based on feature map was proposed, which boosted the effect of distillation by assigning weights to features according to their importance.Experiments were conducted over cifar100 and cifar10 open-source dataset.And various white-box attack algorithms such as FGSM (fast gradient sign method), PGD (project gradient descent) and C＆amp;W (Carlini-Wagner attack) were applied to test the experimental results.The accuracy of the proposed method on Cifar10 clean samples exceeds that of adversarial training and is close to the accuracy of the model trained on clean samples.Under the PGD attack of L2 distance, the efficiency of the proposed method is close to that of adversarial training, which is significantly higher than that of normal training.Moreover, the proposed method is a light-weight adversarial defense method with low learning cost.The computing power requirement is far less than that of adversarial training even if optimization schemes such as attention mechanism and multi-dimensional feature map are added.Knowledge distillation can learn the decision-making experience of normal samples and extract robust features as a neural network learning scheme.It uses a small amount of data to generate accurate and robust models, improves generalization, and reduces the cost of adversarial training.http://www.infocomm-journal.com/cjnis/CN/10.11959/j.issn.2096-109x.2022012deep learningadversarial examples defenseknowledge distillationmulti-dimensional feature maps |
| spellingShingle | Baolin QIU, Ping YI Adversarial examples defense method based on multi-dimensional feature maps knowledge distillation 网络与信息安全学报 deep learning adversarial examples defense knowledge distillation multi-dimensional feature maps |
| title | Adversarial examples defense method based on multi-dimensional feature maps knowledge distillation |
| title_full | Adversarial examples defense method based on multi-dimensional feature maps knowledge distillation |
| title_fullStr | Adversarial examples defense method based on multi-dimensional feature maps knowledge distillation |
| title_full_unstemmed | Adversarial examples defense method based on multi-dimensional feature maps knowledge distillation |
| title_short | Adversarial examples defense method based on multi-dimensional feature maps knowledge distillation |
| title_sort | adversarial examples defense method based on multi dimensional feature maps knowledge distillation |
| topic | deep learning adversarial examples defense knowledge distillation multi-dimensional feature maps |
| url | http://www.infocomm-journal.com/cjnis/CN/10.11959/j.issn.2096-109x.2022012 |
| work_keys_str_mv | AT baolinqiupingyi adversarialexamplesdefensemethodbasedonmultidimensionalfeaturemapsknowledgedistillation |