Offense and defence against adversarial sample: A reinforcement learning method in energy trading market
The energy trading market that can support free bidding among electricity users is currently the key method in smart grid demand response. Reinforcement learning is used to formulate optimal strategies for them to obtain optimal strategies. Non-etheless, the security problem raised by artificial int...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-01-01
|
| Series: | Frontiers in Energy Research |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fenrg.2022.1071973/full |
| _version_ | 1797955276233506816 |
|---|---|
| author | Donghe Li Qingyu Yang Qingyu Yang Linyue Ma Zhenhua Peng Xiao Liao |
| author_facet | Donghe Li Qingyu Yang Qingyu Yang Linyue Ma Zhenhua Peng Xiao Liao |
| author_sort | Donghe Li |
| collection | DOAJ |
| description | The energy trading market that can support free bidding among electricity users is currently the key method in smart grid demand response. Reinforcement learning is used to formulate optimal strategies for them to obtain optimal strategies. Non-etheless, the security problem raised by artificial intelligence technology has been paid more and more attention. For example, the neural network has been proved to be able to resist adversarial example attacks, thus affecting its training results. Considering that reinforcement learning is also widely used for training by neural networks, the security problem can not be ignored, especially in scenarios with high security requirements such as smart grids. To this end, we study the security issues in reinforcement learning-based bidding strategy method facing by the adversarial example. First of all, regarding to the electric vehicle double auction market, we formalize the bidding decision problem of EVs into a Markov Decision Process, so that reinforcement learning is used to solve this problem. Secondly, from the perspective of attackers, we have designed a local Fast Gradient Sign Method which affects the environment and the results of reinforcement learning by changing its own bidding. Then, from the perspective of the defender, we designed a reinforcement learning training network containing an attack identifier based on the deep neural network, so as to identify malicious injection attacks to resist against adversarial attacks. Finally, comprehensive simulations are conducted to verify our proposed method. The results shows that, our proposed attack method will reduce the auction profit by influencing reinforcement learning algorithm, and the protect method will be able to completely resist such attacks. |
| first_indexed | 2024-04-10T23:31:36Z |
| format | Article |
| id | doaj.art-44bda50b12fe4e4a8b425156291ff68c |
| institution | Directory Open Access Journal |
| issn | 2296-598X |
| language | English |
| last_indexed | 2024-04-10T23:31:36Z |
| publishDate | 2023-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Energy Research |
| spelling | doaj.art-44bda50b12fe4e4a8b425156291ff68c2023-01-12T06:04:45ZengFrontiers Media S.A.Frontiers in Energy Research2296-598X2023-01-011010.3389/fenrg.2022.10719731071973Offense and defence against adversarial sample: A reinforcement learning method in energy trading marketDonghe Li0Qingyu Yang1Qingyu Yang2Linyue Ma3Zhenhua Peng4Xiao Liao5School of Automation Science and Engineering, Xi’an Jiaotong University, Xi'an, ChinaSchool of Automation Science and Engineering, Xi’an Jiaotong University, Xi'an, ChinaState Key Laboratory Manufacturing System Engineering, Xi’an Jiaotong University, Xi'an, ChinaState Grid Information and Telecommunication Group Co., LTD, Beijing, ChinaSchool of Automation Science and Engineering, Xi’an Jiaotong University, Xi'an, ChinaState Grid Information and Telecommunication Group Co., LTD, Beijing, ChinaThe energy trading market that can support free bidding among electricity users is currently the key method in smart grid demand response. Reinforcement learning is used to formulate optimal strategies for them to obtain optimal strategies. Non-etheless, the security problem raised by artificial intelligence technology has been paid more and more attention. For example, the neural network has been proved to be able to resist adversarial example attacks, thus affecting its training results. Considering that reinforcement learning is also widely used for training by neural networks, the security problem can not be ignored, especially in scenarios with high security requirements such as smart grids. To this end, we study the security issues in reinforcement learning-based bidding strategy method facing by the adversarial example. First of all, regarding to the electric vehicle double auction market, we formalize the bidding decision problem of EVs into a Markov Decision Process, so that reinforcement learning is used to solve this problem. Secondly, from the perspective of attackers, we have designed a local Fast Gradient Sign Method which affects the environment and the results of reinforcement learning by changing its own bidding. Then, from the perspective of the defender, we designed a reinforcement learning training network containing an attack identifier based on the deep neural network, so as to identify malicious injection attacks to resist against adversarial attacks. Finally, comprehensive simulations are conducted to verify our proposed method. The results shows that, our proposed attack method will reduce the auction profit by influencing reinforcement learning algorithm, and the protect method will be able to completely resist such attacks.https://www.frontiersin.org/articles/10.3389/fenrg.2022.1071973/fulldouble auctionmarkov decision processreinforcement learningadversarial examplefast gradient sign methodadversarial example detection |
| spellingShingle | Donghe Li Qingyu Yang Qingyu Yang Linyue Ma Zhenhua Peng Xiao Liao Offense and defence against adversarial sample: A reinforcement learning method in energy trading market Frontiers in Energy Research double auction markov decision process reinforcement learning adversarial example fast gradient sign method adversarial example detection |
| title | Offense and defence against adversarial sample: A reinforcement learning method in energy trading market |
| title_full | Offense and defence against adversarial sample: A reinforcement learning method in energy trading market |
| title_fullStr | Offense and defence against adversarial sample: A reinforcement learning method in energy trading market |
| title_full_unstemmed | Offense and defence against adversarial sample: A reinforcement learning method in energy trading market |
| title_short | Offense and defence against adversarial sample: A reinforcement learning method in energy trading market |
| title_sort | offense and defence against adversarial sample a reinforcement learning method in energy trading market |
| topic | double auction markov decision process reinforcement learning adversarial example fast gradient sign method adversarial example detection |
| url | https://www.frontiersin.org/articles/10.3389/fenrg.2022.1071973/full |
| work_keys_str_mv | AT dongheli offenseanddefenceagainstadversarialsampleareinforcementlearningmethodinenergytradingmarket AT qingyuyang offenseanddefenceagainstadversarialsampleareinforcementlearningmethodinenergytradingmarket AT qingyuyang offenseanddefenceagainstadversarialsampleareinforcementlearningmethodinenergytradingmarket AT linyuema offenseanddefenceagainstadversarialsampleareinforcementlearningmethodinenergytradingmarket AT zhenhuapeng offenseanddefenceagainstadversarialsampleareinforcementlearningmethodinenergytradingmarket AT xiaoliao offenseanddefenceagainstadversarialsampleareinforcementlearningmethodinenergytradingmarket |