Effective Learning in Non-Stationary Multiagent Environments
Multiagent reinforcement learning (MARL) provides a principled framework for a group of artificial intelligence agents to learn collaborative and/or competitive behaviors at the level of human experts. Multiagent learning settings inherently solve much more complex problems than single-agent learnin...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2023
|
| Online Access: | https://hdl.handle.net/1721.1/150177 |
| _version_ | 1826215896653234176 |
|---|---|
| author | Kim, Dong Ki |
| author2 | How, Jonathan P. |
| author_facet | How, Jonathan P. Kim, Dong Ki |
| author_sort | Kim, Dong Ki |
| collection | MIT |
| description | Multiagent reinforcement learning (MARL) provides a principled framework for a group of artificial intelligence agents to learn collaborative and/or competitive behaviors at the level of human experts. Multiagent learning settings inherently solve much more complex problems than single-agent learning because an agent interacts both with the environment and other agents. In particular, multiple agents simultaneously learn in MARL, leading to natural non-stationarity in the experiences encountered and thus requiring each agent to its behavior with respect to potentially large changes in other agents' policies. This thesis aims to address the non-stationarity challenge in multiagent learning from three important topics: 1) adaptation, 2) convergence, and 3) state space. The first topic answers how an agent can learn effective adaptation strategies concerning other agents' changing policies by developing a new meta-learning framework. The second topic answers how agents can adapt and influence the joint learning process such that policies converge to more desirable limiting behaviors by the end of learning based on a new game-theoretical solution concept.
Lastly, the last topic answers how state space size can be reduced based on knowledge sharing and context-specific abstraction such that the learning complexity is less affected by non-stationarity. In summary, this thesis develops theoretical and algorithmic contributions to provide principled answers to the aforementioned topics on non-stationarity. The developed algorithms in this thesis demonstrate their effectiveness in a diverse suite of multiagent benchmark domains, including the full spectrum of mixed incentive, competitive, and cooperative environments. |
| first_indexed | 2024-09-23T16:38:46Z |
| format | Thesis |
| id | mit-1721.1/150177 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T16:38:46Z |
| publishDate | 2023 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1501772023-04-01T03:03:34Z Effective Learning in Non-Stationary Multiagent Environments Kim, Dong Ki How, Jonathan P. Agrawal, Pulkit Foerster, Jakob N. Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Multiagent reinforcement learning (MARL) provides a principled framework for a group of artificial intelligence agents to learn collaborative and/or competitive behaviors at the level of human experts. Multiagent learning settings inherently solve much more complex problems than single-agent learning because an agent interacts both with the environment and other agents. In particular, multiple agents simultaneously learn in MARL, leading to natural non-stationarity in the experiences encountered and thus requiring each agent to its behavior with respect to potentially large changes in other agents' policies. This thesis aims to address the non-stationarity challenge in multiagent learning from three important topics: 1) adaptation, 2) convergence, and 3) state space. The first topic answers how an agent can learn effective adaptation strategies concerning other agents' changing policies by developing a new meta-learning framework. The second topic answers how agents can adapt and influence the joint learning process such that policies converge to more desirable limiting behaviors by the end of learning based on a new game-theoretical solution concept. Lastly, the last topic answers how state space size can be reduced based on knowledge sharing and context-specific abstraction such that the learning complexity is less affected by non-stationarity. In summary, this thesis develops theoretical and algorithmic contributions to provide principled answers to the aforementioned topics on non-stationarity. The developed algorithms in this thesis demonstrate their effectiveness in a diverse suite of multiagent benchmark domains, including the full spectrum of mixed incentive, competitive, and cooperative environments. Ph.D. 2023-03-31T14:37:46Z 2023-03-31T14:37:46Z 2023-02 2023-02-15T14:05:24.284Z Thesis https://hdl.handle.net/1721.1/150177 In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Kim, Dong Ki Effective Learning in Non-Stationary Multiagent Environments |
| title | Effective Learning in Non-Stationary Multiagent Environments |
| title_full | Effective Learning in Non-Stationary Multiagent Environments |
| title_fullStr | Effective Learning in Non-Stationary Multiagent Environments |
| title_full_unstemmed | Effective Learning in Non-Stationary Multiagent Environments |
| title_short | Effective Learning in Non-Stationary Multiagent Environments |
| title_sort | effective learning in non stationary multiagent environments |
| url | https://hdl.handle.net/1721.1/150177 |
| work_keys_str_mv | AT kimdongki effectivelearninginnonstationarymultiagentenvironments |