eDA3-X: Distributed Attentional Actor Architecture for Interpretability of Coordinated Behaviors in Multi-Agent Systems
In this paper, we propose an enhanced version of the distributed attentional actor architecture (eDA3-X) for model-free reinforcement learning. This architecture is designed to facilitate the interpretability of learned coordinated behaviors in multi-agent systems through the use of a saliency vecto...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/14/8454 |
| _version_ | 1797590381803601920 |
|---|---|
| author | Yoshinari Motokawa Toshiharu Sugawara |
| author_facet | Yoshinari Motokawa Toshiharu Sugawara |
| author_sort | Yoshinari Motokawa |
| collection | DOAJ |
| description | In this paper, we propose an enhanced version of the distributed attentional actor architecture (eDA3-X) for model-free reinforcement learning. This architecture is designed to facilitate the interpretability of learned coordinated behaviors in multi-agent systems through the use of a saliency vector that captures partial observations of the environment. Our proposed method, in principle, can be integrated with any deep reinforcement learning method, as indicated by <i>X</i>, and can help us identify the information in input data that individual agents attend to during and after training. We then validated eDA3-X through experiments in the object collection game. We also analyzed the relationship between cooperative behaviors and three types of attention heatmaps (standard, positional, and class attentions), which provided insight into the information that the agents consider crucial when making decisions. In addition, we investigated how attention is developed by an agent through training experiences. Our experiments indicate that our approach offers a promising solution for understanding coordinated behaviors in multi-agent reinforcement learning. |
| first_indexed | 2024-03-11T01:19:40Z |
| format | Article |
| id | doaj.art-8d9efa7a731b4ddfadbd8009cedbaa51 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T01:19:40Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-8d9efa7a731b4ddfadbd8009cedbaa512023-11-18T18:13:19ZengMDPI AGApplied Sciences2076-34172023-07-011314845410.3390/app13148454eDA3-X: Distributed Attentional Actor Architecture for Interpretability of Coordinated Behaviors in Multi-Agent SystemsYoshinari Motokawa0Toshiharu Sugawara1Department of Computer Science, Waseda University, Tokyo 169-8555, JapanDepartment of Computer Science, Waseda University, Tokyo 169-8555, JapanIn this paper, we propose an enhanced version of the distributed attentional actor architecture (eDA3-X) for model-free reinforcement learning. This architecture is designed to facilitate the interpretability of learned coordinated behaviors in multi-agent systems through the use of a saliency vector that captures partial observations of the environment. Our proposed method, in principle, can be integrated with any deep reinforcement learning method, as indicated by <i>X</i>, and can help us identify the information in input data that individual agents attend to during and after training. We then validated eDA3-X through experiments in the object collection game. We also analyzed the relationship between cooperative behaviors and three types of attention heatmaps (standard, positional, and class attentions), which provided insight into the information that the agents consider crucial when making decisions. In addition, we investigated how attention is developed by an agent through training experiences. Our experiments indicate that our approach offers a promising solution for understanding coordinated behaviors in multi-agent reinforcement learning.https://www.mdpi.com/2076-3417/13/14/8454multi-agent deep reinforcement learningexplainable reinforcement learningdistributed systemattentional mechanismcoordinationcooperation |
| spellingShingle | Yoshinari Motokawa Toshiharu Sugawara eDA3-X: Distributed Attentional Actor Architecture for Interpretability of Coordinated Behaviors in Multi-Agent Systems Applied Sciences multi-agent deep reinforcement learning explainable reinforcement learning distributed system attentional mechanism coordination cooperation |
| title | eDA3-X: Distributed Attentional Actor Architecture for Interpretability of Coordinated Behaviors in Multi-Agent Systems |
| title_full | eDA3-X: Distributed Attentional Actor Architecture for Interpretability of Coordinated Behaviors in Multi-Agent Systems |
| title_fullStr | eDA3-X: Distributed Attentional Actor Architecture for Interpretability of Coordinated Behaviors in Multi-Agent Systems |
| title_full_unstemmed | eDA3-X: Distributed Attentional Actor Architecture for Interpretability of Coordinated Behaviors in Multi-Agent Systems |
| title_short | eDA3-X: Distributed Attentional Actor Architecture for Interpretability of Coordinated Behaviors in Multi-Agent Systems |
| title_sort | eda3 x distributed attentional actor architecture for interpretability of coordinated behaviors in multi agent systems |
| topic | multi-agent deep reinforcement learning explainable reinforcement learning distributed system attentional mechanism coordination cooperation |
| url | https://www.mdpi.com/2076-3417/13/14/8454 |
| work_keys_str_mv | AT yoshinarimotokawa eda3xdistributedattentionalactorarchitectureforinterpretabilityofcoordinatedbehaviorsinmultiagentsystems AT toshiharusugawara eda3xdistributedattentionalactorarchitectureforinterpretabilityofcoordinatedbehaviorsinmultiagentsystems |