Selective real‐time adversarial perturbations against deep reinforcement learning agents
Abstract Recent work has shown that deep reinforcement learning (DRL) is vulnerable to adversarial attacks, so that exploiting vulnerabilities in DRL systems through adversarial attack techniques has become a necessary prerequisite for building robust DRL systems. Compared to traditional deep learni...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-03-01
|
| Series: | IET Cyber-Physical Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/cps2.12065 |
| _version_ | 1797263682984476672 |
|---|---|
| author | Hongjin Yao Yisheng Li Yunpeng Sun Zhichao Lian |
| author_facet | Hongjin Yao Yisheng Li Yunpeng Sun Zhichao Lian |
| author_sort | Hongjin Yao |
| collection | DOAJ |
| description | Abstract Recent work has shown that deep reinforcement learning (DRL) is vulnerable to adversarial attacks, so that exploiting vulnerabilities in DRL systems through adversarial attack techniques has become a necessary prerequisite for building robust DRL systems. Compared to traditional deep learning systems, DRL systems are characterised by long sequential decisions rather than one‐step decision, so attackers must perform multi‐step attacks on them. To successfully attack a DRL system, the number of attacks must be minimised to avoid detecting by the victim agent and to ensure the effectiveness of the attack. Some selective attack methods proposed in recent researches, that is, attacking an agent at partial time steps, are not applicable to real‐time attack scenarios, although they can avoid detecting by the victim agent. A real‐time selective attack method that is applicable to environments with discrete action spaces is proposed. Firstly, the optimal attack threshold T for performing selective attacks in the environment Env is determined. Then, the observation states corresponding to when the value of the action preference function of the victim agent in multiple eposides exceeds the threshold T are added to the training set according to this threshold. Finally, a universal perturbation is generated based on this training set, and it is used to perform real‐time selective attacks on the victim agent. Comparative experiments show that our attack method can perform real‐time attacks while maintaining the attack effect and stealthiness. |
| first_indexed | 2024-04-25T00:16:54Z |
| format | Article |
| id | doaj.art-a5cf5495005c41dfbcf146138605fbce |
| institution | Directory Open Access Journal |
| issn | 2398-3396 |
| language | English |
| last_indexed | 2024-04-25T00:16:54Z |
| publishDate | 2024-03-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Cyber-Physical Systems |
| spelling | doaj.art-a5cf5495005c41dfbcf146138605fbce2024-03-13T03:51:42ZengWileyIET Cyber-Physical Systems2398-33962024-03-0191414910.1049/cps2.12065Selective real‐time adversarial perturbations against deep reinforcement learning agentsHongjin Yao0Yisheng Li1Yunpeng Sun2Zhichao Lian3School of Computer Science and Technology Nanjing University of Science and Technology Nanjing ChinaSchool of Cyberspace Security Nanjing University of Science and Technology Nanjing ChinaSchool of Cyberspace Security Nanjing University of Science and Technology Nanjing ChinaSchool of Cyberspace Security Nanjing University of Science and Technology Nanjing ChinaAbstract Recent work has shown that deep reinforcement learning (DRL) is vulnerable to adversarial attacks, so that exploiting vulnerabilities in DRL systems through adversarial attack techniques has become a necessary prerequisite for building robust DRL systems. Compared to traditional deep learning systems, DRL systems are characterised by long sequential decisions rather than one‐step decision, so attackers must perform multi‐step attacks on them. To successfully attack a DRL system, the number of attacks must be minimised to avoid detecting by the victim agent and to ensure the effectiveness of the attack. Some selective attack methods proposed in recent researches, that is, attacking an agent at partial time steps, are not applicable to real‐time attack scenarios, although they can avoid detecting by the victim agent. A real‐time selective attack method that is applicable to environments with discrete action spaces is proposed. Firstly, the optimal attack threshold T for performing selective attacks in the environment Env is determined. Then, the observation states corresponding to when the value of the action preference function of the victim agent in multiple eposides exceeds the threshold T are added to the training set according to this threshold. Finally, a universal perturbation is generated based on this training set, and it is used to perform real‐time selective attacks on the victim agent. Comparative experiments show that our attack method can perform real‐time attacks while maintaining the attack effect and stealthiness.https://doi.org/10.1049/cps2.12065decision makinggradient methodsimage classification |
| spellingShingle | Hongjin Yao Yisheng Li Yunpeng Sun Zhichao Lian Selective real‐time adversarial perturbations against deep reinforcement learning agents IET Cyber-Physical Systems decision making gradient methods image classification |
| title | Selective real‐time adversarial perturbations against deep reinforcement learning agents |
| title_full | Selective real‐time adversarial perturbations against deep reinforcement learning agents |
| title_fullStr | Selective real‐time adversarial perturbations against deep reinforcement learning agents |
| title_full_unstemmed | Selective real‐time adversarial perturbations against deep reinforcement learning agents |
| title_short | Selective real‐time adversarial perturbations against deep reinforcement learning agents |
| title_sort | selective real time adversarial perturbations against deep reinforcement learning agents |
| topic | decision making gradient methods image classification |
| url | https://doi.org/10.1049/cps2.12065 |
| work_keys_str_mv | AT hongjinyao selectiverealtimeadversarialperturbationsagainstdeepreinforcementlearningagents AT yishengli selectiverealtimeadversarialperturbationsagainstdeepreinforcementlearningagents AT yunpengsun selectiverealtimeadversarialperturbationsagainstdeepreinforcementlearningagents AT zhichaolian selectiverealtimeadversarialperturbationsagainstdeepreinforcementlearningagents |