Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment
The reinforcement learning problem of complex action control in a multi-player wargame has been a hot research topic in recent years. In this paper, a game system based on turn-based confrontation is designed and implemented with state-of-the-art deep reinforcement learning models. Specifically, we...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-10-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/9/10/1668 |
| _version_ | 1797551153194467328 |
|---|---|
| author | Yuxiang Sun Bo Yuan Tao Zhang Bojian Tang Wanwen Zheng Xianzhong Zhou |
| author_facet | Yuxiang Sun Bo Yuan Tao Zhang Bojian Tang Wanwen Zheng Xianzhong Zhou |
| author_sort | Yuxiang Sun |
| collection | DOAJ |
| description | The reinforcement learning problem of complex action control in a multi-player wargame has been a hot research topic in recent years. In this paper, a game system based on turn-based confrontation is designed and implemented with state-of-the-art deep reinforcement learning models. Specifically, we first design a Q-learning algorithm to achieve intelligent decision-making, which is based on the DQN (Deep Q Network) to model complex game behaviors. Then, an a priori knowledge-based algorithm PK-DQN (Prior Knowledge-Deep Q Network) is introduced to improve the DQN algorithm, which accelerates the convergence speed and stability of the algorithm. The experiments demonstrate the correctness of the PK-DQN algorithm, it is validated, and its performance surpasses the conventional DQN algorithm. Furthermore, the PK-DQN algorithm shows effectiveness in defeating the high level of rule-based opponents, which provides promising results for the exploration of the field of smart chess and intelligent game deduction. |
| first_indexed | 2024-03-10T15:40:40Z |
| format | Article |
| id | doaj.art-a29f317bd74947878446e62423d129b0 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-10T15:40:40Z |
| publishDate | 2020-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-a29f317bd74947878446e62423d129b02023-11-20T16:53:00ZengMDPI AGElectronics2079-92922020-10-01910166810.3390/electronics9101668Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame EnvironmentYuxiang Sun0Bo Yuan1Tao Zhang2Bojian Tang3Wanwen Zheng4Xianzhong Zhou5School of Management and Engineering, Nanjing University, Nanjing 210023, ChinaSchool of Electronics, Computing and Mathematics, University of Derby, Kedleston Rd, Derby DE22 1GB, UKSchool of Management and Engineering, Nanjing University, Nanjing 210023, ChinaSchool of Management and Engineering, Nanjing University, Nanjing 210023, ChinaSchool of Management and Engineering, Nanjing University, Nanjing 210023, ChinaSchool of Management and Engineering, Nanjing University, Nanjing 210023, ChinaThe reinforcement learning problem of complex action control in a multi-player wargame has been a hot research topic in recent years. In this paper, a game system based on turn-based confrontation is designed and implemented with state-of-the-art deep reinforcement learning models. Specifically, we first design a Q-learning algorithm to achieve intelligent decision-making, which is based on the DQN (Deep Q Network) to model complex game behaviors. Then, an a priori knowledge-based algorithm PK-DQN (Prior Knowledge-Deep Q Network) is introduced to improve the DQN algorithm, which accelerates the convergence speed and stability of the algorithm. The experiments demonstrate the correctness of the PK-DQN algorithm, it is validated, and its performance surpasses the conventional DQN algorithm. Furthermore, the PK-DQN algorithm shows effectiveness in defeating the high level of rule-based opponents, which provides promising results for the exploration of the field of smart chess and intelligent game deduction.https://www.mdpi.com/2079-9292/9/10/1668DQN algorithmpolicy modelingprior knowledgeintelligent decision |
| spellingShingle | Yuxiang Sun Bo Yuan Tao Zhang Bojian Tang Wanwen Zheng Xianzhong Zhou Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment Electronics DQN algorithm policy modeling prior knowledge intelligent decision |
| title | Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment |
| title_full | Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment |
| title_fullStr | Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment |
| title_full_unstemmed | Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment |
| title_short | Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment |
| title_sort | research and implementation of intelligent decision based on a priori knowledge and dqn algorithms in wargame environment |
| topic | DQN algorithm policy modeling prior knowledge intelligent decision |
| url | https://www.mdpi.com/2079-9292/9/10/1668 |
| work_keys_str_mv | AT yuxiangsun researchandimplementationofintelligentdecisionbasedonaprioriknowledgeanddqnalgorithmsinwargameenvironment AT boyuan researchandimplementationofintelligentdecisionbasedonaprioriknowledgeanddqnalgorithmsinwargameenvironment AT taozhang researchandimplementationofintelligentdecisionbasedonaprioriknowledgeanddqnalgorithmsinwargameenvironment AT bojiantang researchandimplementationofintelligentdecisionbasedonaprioriknowledgeanddqnalgorithmsinwargameenvironment AT wanwenzheng researchandimplementationofintelligentdecisionbasedonaprioriknowledgeanddqnalgorithmsinwargameenvironment AT xianzhongzhou researchandimplementationofintelligentdecisionbasedonaprioriknowledgeanddqnalgorithmsinwargameenvironment |