Policy Distillation and Value Matching in Multiagent Reinforcement Learning
© 2019 IEEE. Multiagent reinforcement learning (MARL) algorithms have been demonstrated on complex tasks that require the coordination of a team of multiple agents to complete. Existing works have focused on sharing information between agents via centralized critics to stabilize learning or through...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021
|
| Online Access: | https://hdl.handle.net/1721.1/137155 |
| _version_ | 1811082957614481408 |
|---|---|
| author | Wadhwania, Samir Kim, Dong-Ki Omidshafiei, Shayegan How, Jonathan P. |
| author2 | Massachusetts Institute of Technology. Laboratory for Information and Decision Systems |
| author_facet | Massachusetts Institute of Technology. Laboratory for Information and Decision Systems Wadhwania, Samir Kim, Dong-Ki Omidshafiei, Shayegan How, Jonathan P. |
| author_sort | Wadhwania, Samir |
| collection | MIT |
| description | © 2019 IEEE. Multiagent reinforcement learning (MARL) algorithms have been demonstrated on complex tasks that require the coordination of a team of multiple agents to complete. Existing works have focused on sharing information between agents via centralized critics to stabilize learning or through communication to improve performance, but do not generally consider how information can be shared between agents to address the curse of dimensionality in MARL. We posit that a multiagent problem can be decomposed into a multi-task problem where each agent explores a subset of the state space instead of exploring the entire state space. This paper introduces a multiagent actor-critic algorithm for combining knowledge from homogeneous agents through distillation and value-matching that outperforms policy distillation alone and allows further learning in discrete and continuous action spaces. |
| first_indexed | 2024-09-23T12:15:25Z |
| format | Article |
| id | mit-1721.1/137155 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T12:15:25Z |
| publishDate | 2021 |
| publisher | IEEE |
| record_format | dspace |
| spelling | mit-1721.1/1371552023-02-14T19:22:36Z Policy Distillation and Value Matching in Multiagent Reinforcement Learning Wadhwania, Samir Kim, Dong-Ki Omidshafiei, Shayegan How, Jonathan P. Massachusetts Institute of Technology. Laboratory for Information and Decision Systems © 2019 IEEE. Multiagent reinforcement learning (MARL) algorithms have been demonstrated on complex tasks that require the coordination of a team of multiple agents to complete. Existing works have focused on sharing information between agents via centralized critics to stabilize learning or through communication to improve performance, but do not generally consider how information can be shared between agents to address the curse of dimensionality in MARL. We posit that a multiagent problem can be decomposed into a multi-task problem where each agent explores a subset of the state space instead of exploring the entire state space. This paper introduces a multiagent actor-critic algorithm for combining knowledge from homogeneous agents through distillation and value-matching that outperforms policy distillation alone and allows further learning in discrete and continuous action spaces. 2021-11-02T18:16:00Z 2021-11-02T18:16:00Z 2019-11 2021-04-30T13:50:45Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/137155 Wadhwania, Samir, Kim, Dong-Ki, Omidshafiei, Shayegan and How, Jonathan P. 2019. "Policy Distillation and Value Matching in Multiagent Reinforcement Learning." IEEE International Conference on Intelligent Robots and Systems. en 10.1109/iros40897.2019.8967849 IEEE International Conference on Intelligent Robots and Systems Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf IEEE arXiv |
| spellingShingle | Wadhwania, Samir Kim, Dong-Ki Omidshafiei, Shayegan How, Jonathan P. Policy Distillation and Value Matching in Multiagent Reinforcement Learning |
| title | Policy Distillation and Value Matching in Multiagent Reinforcement Learning |
| title_full | Policy Distillation and Value Matching in Multiagent Reinforcement Learning |
| title_fullStr | Policy Distillation and Value Matching in Multiagent Reinforcement Learning |
| title_full_unstemmed | Policy Distillation and Value Matching in Multiagent Reinforcement Learning |
| title_short | Policy Distillation and Value Matching in Multiagent Reinforcement Learning |
| title_sort | policy distillation and value matching in multiagent reinforcement learning |
| url | https://hdl.handle.net/1721.1/137155 |
| work_keys_str_mv | AT wadhwaniasamir policydistillationandvaluematchinginmultiagentreinforcementlearning AT kimdongki policydistillationandvaluematchinginmultiagentreinforcementlearning AT omidshafieishayegan policydistillationandvaluematchinginmultiagentreinforcementlearning AT howjonathanp policydistillationandvaluematchinginmultiagentreinforcementlearning |