Analysing factorizations of action-value networks for cooperative multi-agent reinforcement learning
Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. However, given the lack of theoretical insight, it remains unclear what the employed neural networks are learning, or how we should enhance their learnin...
| Autors principals: | , , , |
|---|---|
| Format: | Journal article |
| Idioma: | English |
| Publicat: |
Springer
2021
|
| _version_ | 1826276977314627584 |
|---|---|
| author | Castellini, J Oliehoek, FA Savani, R Whiteson, S |
| author_facet | Castellini, J Oliehoek, FA Savani, R Whiteson, S |
| author_sort | Castellini, J |
| collection | OXFORD |
| description | Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. However, given the lack of theoretical insight, it remains unclear what the employed neural networks are learning, or how we should enhance their learning power to address the problems on which they fail. In this work, we empirically investigate the learning power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results extend those in Castellini et al. (Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS’19.International Foundation for Autonomous Agents and Multiagent Systems, pp 1862–1864, 2019) and quantify how well various approaches can represent the requisite value functions, and help us identify the reasons that can impede good performance, like sparsity of the values or too tight coordination requirements. |
| first_indexed | 2024-03-06T23:21:57Z |
| format | Journal article |
| id | oxford-uuid:6906d9db-e09b-494c-b1ee-cb1a21c7de9c |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T23:21:57Z |
| publishDate | 2021 |
| publisher | Springer |
| record_format | dspace |
| spelling | oxford-uuid:6906d9db-e09b-494c-b1ee-cb1a21c7de9c2022-03-26T18:48:50ZAnalysing factorizations of action-value networks for cooperative multi-agent reinforcement learningJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:6906d9db-e09b-494c-b1ee-cb1a21c7de9cEnglishSymplectic ElementsSpringer2021Castellini, JOliehoek, FASavani, RWhiteson, SRecent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. However, given the lack of theoretical insight, it remains unclear what the employed neural networks are learning, or how we should enhance their learning power to address the problems on which they fail. In this work, we empirically investigate the learning power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results extend those in Castellini et al. (Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS’19.International Foundation for Autonomous Agents and Multiagent Systems, pp 1862–1864, 2019) and quantify how well various approaches can represent the requisite value functions, and help us identify the reasons that can impede good performance, like sparsity of the values or too tight coordination requirements. |
| spellingShingle | Castellini, J Oliehoek, FA Savani, R Whiteson, S Analysing factorizations of action-value networks for cooperative multi-agent reinforcement learning |
| title | Analysing factorizations of action-value networks for cooperative multi-agent reinforcement learning |
| title_full | Analysing factorizations of action-value networks for cooperative multi-agent reinforcement learning |
| title_fullStr | Analysing factorizations of action-value networks for cooperative multi-agent reinforcement learning |
| title_full_unstemmed | Analysing factorizations of action-value networks for cooperative multi-agent reinforcement learning |
| title_short | Analysing factorizations of action-value networks for cooperative multi-agent reinforcement learning |
| title_sort | analysing factorizations of action value networks for cooperative multi agent reinforcement learning |
| work_keys_str_mv | AT castellinij analysingfactorizationsofactionvaluenetworksforcooperativemultiagentreinforcementlearning AT oliehoekfa analysingfactorizationsofactionvaluenetworksforcooperativemultiagentreinforcementlearning AT savanir analysingfactorizationsofactionvaluenetworksforcooperativemultiagentreinforcementlearning AT whitesons analysingfactorizationsofactionvaluenetworksforcooperativemultiagentreinforcementlearning |