Attacks based on malicious perturbations on image processing systems and defense methods against them
Systems implementing artificial intelligence technologies have become widespread due to their effectiveness in solving various applied tasks including computer vision. Image processing through neural networks is also used in securitycritical systems. At the same time, the use of artificial intellige...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Saint Petersburg National Research University of Information Technologies, Mechanics and Optics (ITMO University)
2023-08-01
|
| Series: | Naučno-tehničeskij Vestnik Informacionnyh Tehnologij, Mehaniki i Optiki |
| Subjects: | |
| Online Access: | https://ntv.ifmo.ru/file/article/22193.pdf |
| _version_ | 1797740075957616640 |
|---|---|
| author | Dmitry A. Esipov Abdulhamid Y. Buchaev Akylzhan Kerimbay Yana V. Puzikova Semen K. Saidumarov Nikita S. Sulimenko Ilya Yu. Popov Nikolay S. Karmanovskiy |
| author_facet | Dmitry A. Esipov Abdulhamid Y. Buchaev Akylzhan Kerimbay Yana V. Puzikova Semen K. Saidumarov Nikita S. Sulimenko Ilya Yu. Popov Nikolay S. Karmanovskiy |
| author_sort | Dmitry A. Esipov |
| collection | DOAJ |
| description | Systems implementing artificial intelligence technologies have become widespread due to their effectiveness in solving various applied tasks including computer vision. Image processing through neural networks is also used in securitycritical systems. At the same time, the use of artificial intelligence is associated with characteristic threats including disruption of machine learning models. The phenomenon of triggering an incorrect neural network response by introducing perturbations that are visually imperceptible to a person was first described and attracted the attention of researchers in 2013. Methods of attacks on neural networks based on malicious perturbations have been continuously improved, ways of disrupting the operation of neural networks in processing various types of data and tasks of the target model have been proposed. The threat of disrupting the functioning of neural networks through these attacks has become a significant problem for systems implementing artificial intelligence technologies. Thus, research in the field of countering attacks based on malicious perturbations is very relevant. This article describes current attacks, provides an overview and comparative analysis of such attacks on image processing systems based on artificial intelligence. Approaches to the classification of attacks based on malicious perturbations are formulated. Defense methods against such attacks are considered, their shortcomings are revealed. The limitations of the applied defense methods that reduce the effectiveness of counteraction to attacks are shown. Approaches and practical measures to detect and eliminate harmful disturbances are proposed. |
| first_indexed | 2024-03-12T14:07:19Z |
| format | Article |
| id | doaj.art-0aa74a69458f4ea8aeea0bd250cf7b1b |
| institution | Directory Open Access Journal |
| issn | 2226-1494 2500-0373 |
| language | English |
| last_indexed | 2024-03-12T14:07:19Z |
| publishDate | 2023-08-01 |
| publisher | Saint Petersburg National Research University of Information Technologies, Mechanics and Optics (ITMO University) |
| record_format | Article |
| series | Naučno-tehničeskij Vestnik Informacionnyh Tehnologij, Mehaniki i Optiki |
| spelling | doaj.art-0aa74a69458f4ea8aeea0bd250cf7b1b2023-08-21T11:29:29ZengSaint Petersburg National Research University of Information Technologies, Mechanics and Optics (ITMO University)Naučno-tehničeskij Vestnik Informacionnyh Tehnologij, Mehaniki i Optiki2226-14942500-03732023-08-0123472073310.17586/2226-1494-2023-23-4-720-733Attacks based on malicious perturbations on image processing systems and defense methods against themDmitry A. Esipov0https://orcid.org/0000-0003-4467-5117Abdulhamid Y. Buchaev1https://orcid.org/0009-0001-1058-9125Akylzhan Kerimbay2https://orcid.org/0009-0009-9945-9906Yana V. Puzikova3https://orcid.org/0009-0007-7604-3022Semen K. Saidumarov4https://orcid.org/0009-0008-0774-9803Nikita S. Sulimenko5https://orcid.org/0009-0007-3218-9249Ilya Yu. Popov6https://orcid.org/0000-0002-6407-7934Nikolay S. Karmanovskiy7https://orcid.org/0000-0002-0533-9893Engineer, ITMO University, Saint Petersburg, 197101, Russian FederationEngineer, ITMO University, Saint Petersburg, 197101, Russian Federation, sc 57219568840Engineer, ITMO University, Saint Petersburg, 197101, Russian FederationEngineer, ITMO University, Saint Petersburg, 197101, Russian FederationStudent, ITMO University, Saint Petersburg, 197101, Russian FederationStudent, ITMO University, Saint Petersburg, 197101, Russian FederationPhD, Associate Professor, ITMO University, Saint Petersburg, 197101, Russian Federation, sc 57202195632PhD, Associate Professor, Associate Professor, ITMO University, Saint Petersburg, 197101, Russian Federation, sc 57192385103Systems implementing artificial intelligence technologies have become widespread due to their effectiveness in solving various applied tasks including computer vision. Image processing through neural networks is also used in securitycritical systems. At the same time, the use of artificial intelligence is associated with characteristic threats including disruption of machine learning models. The phenomenon of triggering an incorrect neural network response by introducing perturbations that are visually imperceptible to a person was first described and attracted the attention of researchers in 2013. Methods of attacks on neural networks based on malicious perturbations have been continuously improved, ways of disrupting the operation of neural networks in processing various types of data and tasks of the target model have been proposed. The threat of disrupting the functioning of neural networks through these attacks has become a significant problem for systems implementing artificial intelligence technologies. Thus, research in the field of countering attacks based on malicious perturbations is very relevant. This article describes current attacks, provides an overview and comparative analysis of such attacks on image processing systems based on artificial intelligence. Approaches to the classification of attacks based on malicious perturbations are formulated. Defense methods against such attacks are considered, their shortcomings are revealed. The limitations of the applied defense methods that reduce the effectiveness of counteraction to attacks are shown. Approaches and practical measures to detect and eliminate harmful disturbances are proposed.https://ntv.ifmo.ru/file/article/22193.pdfartificial intelligenceartificial neural networkimage processingadversarial attackbackdoor embeddingadversarial perturbationadversarial learningdefense distillationfeature squeezingcertified defensedata preprocessing |
| spellingShingle | Dmitry A. Esipov Abdulhamid Y. Buchaev Akylzhan Kerimbay Yana V. Puzikova Semen K. Saidumarov Nikita S. Sulimenko Ilya Yu. Popov Nikolay S. Karmanovskiy Attacks based on malicious perturbations on image processing systems and defense methods against them Naučno-tehničeskij Vestnik Informacionnyh Tehnologij, Mehaniki i Optiki artificial intelligence artificial neural network image processing adversarial attack backdoor embedding adversarial perturbation adversarial learning defense distillation feature squeezing certified defense data preprocessing |
| title | Attacks based on malicious perturbations on image processing systems and defense methods against them |
| title_full | Attacks based on malicious perturbations on image processing systems and defense methods against them |
| title_fullStr | Attacks based on malicious perturbations on image processing systems and defense methods against them |
| title_full_unstemmed | Attacks based on malicious perturbations on image processing systems and defense methods against them |
| title_short | Attacks based on malicious perturbations on image processing systems and defense methods against them |
| title_sort | attacks based on malicious perturbations on image processing systems and defense methods against them |
| topic | artificial intelligence artificial neural network image processing adversarial attack backdoor embedding adversarial perturbation adversarial learning defense distillation feature squeezing certified defense data preprocessing |
| url | https://ntv.ifmo.ru/file/article/22193.pdf |
| work_keys_str_mv | AT dmitryaesipov attacksbasedonmaliciousperturbationsonimageprocessingsystemsanddefensemethodsagainstthem AT abdulhamidybuchaev attacksbasedonmaliciousperturbationsonimageprocessingsystemsanddefensemethodsagainstthem AT akylzhankerimbay attacksbasedonmaliciousperturbationsonimageprocessingsystemsanddefensemethodsagainstthem AT yanavpuzikova attacksbasedonmaliciousperturbationsonimageprocessingsystemsanddefensemethodsagainstthem AT semenksaidumarov attacksbasedonmaliciousperturbationsonimageprocessingsystemsanddefensemethodsagainstthem AT nikitassulimenko attacksbasedonmaliciousperturbationsonimageprocessingsystemsanddefensemethodsagainstthem AT ilyayupopov attacksbasedonmaliciousperturbationsonimageprocessingsystemsanddefensemethodsagainstthem AT nikolayskarmanovskiy attacksbasedonmaliciousperturbationsonimageprocessingsystemsanddefensemethodsagainstthem |