Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents
Performance of state-of-the art offline and model-based reinforcement learning (RL) algorithms deteriorates significantly when subjected to severe data scarcity and the presence of heterogeneous agents. In this work, we propose a model-based offline RL method to approach this setting. Using all avai...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2022
|
| Online Access: | https://hdl.handle.net/1721.1/139143 |
| _version_ | 1826192706900066304 |
|---|---|
| author | Alumootil, Varkey |
| author2 | Shah, Devavrat |
| author_facet | Shah, Devavrat Alumootil, Varkey |
| author_sort | Alumootil, Varkey |
| collection | MIT |
| description | Performance of state-of-the art offline and model-based reinforcement learning (RL) algorithms deteriorates significantly when subjected to severe data scarcity and the presence of heterogeneous agents. In this work, we propose a model-based offline RL method to approach this setting. Using all available data from the various agents, we construct personalized simulators for each individual agent, which are then used to train RL policies. We do so by modeling the transition dynamics of the agents as a low rank tensor decomposition of latent factors associated with agents, states, and actions. We perform experiments on various benchmark environments and demonstrate improvement over existing offline approaches in the scarce data regime. |
| first_indexed | 2024-09-23T09:28:01Z |
| format | Thesis |
| id | mit-1721.1/139143 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T09:28:01Z |
| publishDate | 2022 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1391432022-01-15T04:00:23Z Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents Alumootil, Varkey Shah, Devavrat Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Performance of state-of-the art offline and model-based reinforcement learning (RL) algorithms deteriorates significantly when subjected to severe data scarcity and the presence of heterogeneous agents. In this work, we propose a model-based offline RL method to approach this setting. Using all available data from the various agents, we construct personalized simulators for each individual agent, which are then used to train RL policies. We do so by modeling the transition dynamics of the agents as a low rank tensor decomposition of latent factors associated with agents, states, and actions. We perform experiments on various benchmark environments and demonstrate improvement over existing offline approaches in the scarce data regime. M.Eng. 2022-01-14T14:52:30Z 2022-01-14T14:52:30Z 2021-06 2021-06-17T20:12:49.354Z Thesis https://hdl.handle.net/1721.1/139143 In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Alumootil, Varkey Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents |
| title | Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents |
| title_full | Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents |
| title_fullStr | Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents |
| title_full_unstemmed | Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents |
| title_short | Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents |
| title_sort | data efficient offline reinforcement learning with heterogeneous agents |
| url | https://hdl.handle.net/1721.1/139143 |
| work_keys_str_mv | AT alumootilvarkey dataefficientofflinereinforcementlearningwithheterogeneousagents |