KnowRU: Knowledge Reuse via Knowledge Distillation in Multi-Agent Reinforcement Learning
Recently, deep reinforcement learning (RL) algorithms have achieved significant progress in the multi-agent domain. However, training for increasingly complex tasks would be time-consuming and resource intensive. To alleviate this problem, efficient leveraging of historical experience is essential,...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-08-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/23/8/1043 |
| _version_ | 1797523887310766080 |
|---|---|
| author | Zijian Gao Kele Xu Bo Ding Huaimin Wang |
| author_facet | Zijian Gao Kele Xu Bo Ding Huaimin Wang |
| author_sort | Zijian Gao |
| collection | DOAJ |
| description | Recently, deep reinforcement learning (RL) algorithms have achieved significant progress in the multi-agent domain. However, training for increasingly complex tasks would be time-consuming and resource intensive. To alleviate this problem, efficient leveraging of historical experience is essential, which is under-explored in previous studies because most existing methods fail to achieve this goal in a continuously dynamic system owing to their complicated design. In this paper, we propose a method for knowledge reuse called “KnowRU”, which can be easily deployed in the majority of multi-agent reinforcement learning (MARL) algorithms without requiring complicated hand-coded design. We employ the knowledge distillation paradigm to transfer knowledge among agents to shorten the training phase for new tasks while improving the asymptotic performance of agents. To empirically demonstrate the robustness and effectiveness of KnowRU, we perform extensive experiments on state-of-the-art MARL algorithms in collaborative and competitive scenarios. The results show that KnowRU outperforms recently reported methods and not only successfully accelerates the training phase, but also improves the training performance, emphasizing the importance of the proposed knowledge reuse for MARL. |
| first_indexed | 2024-03-10T08:49:30Z |
| format | Article |
| id | doaj.art-2a6df7a8341741e8bfd51485f6b67aef |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T08:49:30Z |
| publishDate | 2021-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-2a6df7a8341741e8bfd51485f6b67aef2023-11-22T07:35:29ZengMDPI AGEntropy1099-43002021-08-01238104310.3390/e23081043KnowRU: Knowledge Reuse via Knowledge Distillation in Multi-Agent Reinforcement LearningZijian Gao0Kele Xu1Bo Ding2Huaimin Wang3College of Computer, National University of Defense Technology, Changsha 410000, ChinaCollege of Computer, National University of Defense Technology, Changsha 410000, ChinaCollege of Computer, National University of Defense Technology, Changsha 410000, ChinaCollege of Computer, National University of Defense Technology, Changsha 410000, ChinaRecently, deep reinforcement learning (RL) algorithms have achieved significant progress in the multi-agent domain. However, training for increasingly complex tasks would be time-consuming and resource intensive. To alleviate this problem, efficient leveraging of historical experience is essential, which is under-explored in previous studies because most existing methods fail to achieve this goal in a continuously dynamic system owing to their complicated design. In this paper, we propose a method for knowledge reuse called “KnowRU”, which can be easily deployed in the majority of multi-agent reinforcement learning (MARL) algorithms without requiring complicated hand-coded design. We employ the knowledge distillation paradigm to transfer knowledge among agents to shorten the training phase for new tasks while improving the asymptotic performance of agents. To empirically demonstrate the robustness and effectiveness of KnowRU, we perform extensive experiments on state-of-the-art MARL algorithms in collaborative and competitive scenarios. The results show that KnowRU outperforms recently reported methods and not only successfully accelerates the training phase, but also improves the training performance, emphasizing the importance of the proposed knowledge reuse for MARL.https://www.mdpi.com/1099-4300/23/8/1043multi-agent reinforcement learningknowledge reuseknowledge distillation |
| spellingShingle | Zijian Gao Kele Xu Bo Ding Huaimin Wang KnowRU: Knowledge Reuse via Knowledge Distillation in Multi-Agent Reinforcement Learning Entropy multi-agent reinforcement learning knowledge reuse knowledge distillation |
| title | KnowRU: Knowledge Reuse via Knowledge Distillation in Multi-Agent Reinforcement Learning |
| title_full | KnowRU: Knowledge Reuse via Knowledge Distillation in Multi-Agent Reinforcement Learning |
| title_fullStr | KnowRU: Knowledge Reuse via Knowledge Distillation in Multi-Agent Reinforcement Learning |
| title_full_unstemmed | KnowRU: Knowledge Reuse via Knowledge Distillation in Multi-Agent Reinforcement Learning |
| title_short | KnowRU: Knowledge Reuse via Knowledge Distillation in Multi-Agent Reinforcement Learning |
| title_sort | knowru knowledge reuse via knowledge distillation in multi agent reinforcement learning |
| topic | multi-agent reinforcement learning knowledge reuse knowledge distillation |
| url | https://www.mdpi.com/1099-4300/23/8/1043 |
| work_keys_str_mv | AT zijiangao knowruknowledgereuseviaknowledgedistillationinmultiagentreinforcementlearning AT kelexu knowruknowledgereuseviaknowledgedistillationinmultiagentreinforcementlearning AT boding knowruknowledgereuseviaknowledgedistillationinmultiagentreinforcementlearning AT huaiminwang knowruknowledgereuseviaknowledgedistillationinmultiagentreinforcementlearning |