A Unifying Framework for Reinforcement Learning and Planning
Sequential decision making, commonly formalized as optimization of a Markov Decision Process, is a key challenge in artificial intelligence. Two successful approaches to MDP optimization are reinforcement learning and planning, which both largely have their own research communities. However, if both...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-07-01
|
| Series: | Frontiers in Artificial Intelligence |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frai.2022.908353/full |
| _version_ | 1818518958479769600 |
|---|---|
| author | Thomas M. Moerland Joost Broekens Aske Plaat Catholijn M. Jonker Catholijn M. Jonker |
| author_facet | Thomas M. Moerland Joost Broekens Aske Plaat Catholijn M. Jonker Catholijn M. Jonker |
| author_sort | Thomas M. Moerland |
| collection | DOAJ |
| description | Sequential decision making, commonly formalized as optimization of a Markov Decision Process, is a key challenge in artificial intelligence. Two successful approaches to MDP optimization are reinforcement learning and planning, which both largely have their own research communities. However, if both research fields solve the same problem, then we might be able to disentangle the common factors in their solution approaches. Therefore, this paper presents a unifying algorithmic framework for reinforcement learning and planning (FRAP), which identifies underlying dimensions on which MDP planning and learning algorithms have to decide. At the end of the paper, we compare a variety of well-known planning, model-free and model-based RL algorithms along these dimensions. Altogether, the framework may help provide deeper insight in the algorithmic design space of planning and reinforcement learning. |
| first_indexed | 2024-12-11T01:17:21Z |
| format | Article |
| id | doaj.art-1964865081014a27b68c9d4d7dda37c8 |
| institution | Directory Open Access Journal |
| issn | 2624-8212 |
| language | English |
| last_indexed | 2024-12-11T01:17:21Z |
| publishDate | 2022-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Artificial Intelligence |
| spelling | doaj.art-1964865081014a27b68c9d4d7dda37c82022-12-22T01:25:50ZengFrontiers Media S.A.Frontiers in Artificial Intelligence2624-82122022-07-01510.3389/frai.2022.908353908353A Unifying Framework for Reinforcement Learning and PlanningThomas M. Moerland0Joost Broekens1Aske Plaat2Catholijn M. Jonker3Catholijn M. Jonker4Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, NetherlandsLeiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, NetherlandsLeiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, NetherlandsLeiden Institute of Advanced Computer Science (LIACS), Leiden University, Leiden, NetherlandsInteractive Intelligence, Delft University of Technology, Delft, NetherlandsSequential decision making, commonly formalized as optimization of a Markov Decision Process, is a key challenge in artificial intelligence. Two successful approaches to MDP optimization are reinforcement learning and planning, which both largely have their own research communities. However, if both research fields solve the same problem, then we might be able to disentangle the common factors in their solution approaches. Therefore, this paper presents a unifying algorithmic framework for reinforcement learning and planning (FRAP), which identifies underlying dimensions on which MDP planning and learning algorithms have to decide. At the end of the paper, we compare a variety of well-known planning, model-free and model-based RL algorithms along these dimensions. Altogether, the framework may help provide deeper insight in the algorithmic design space of planning and reinforcement learning.https://www.frontiersin.org/articles/10.3389/frai.2022.908353/fullplanningreinforcement learningmodel-based reinforcement learningframeworkoverviewsynthesis |
| spellingShingle | Thomas M. Moerland Joost Broekens Aske Plaat Catholijn M. Jonker Catholijn M. Jonker A Unifying Framework for Reinforcement Learning and Planning Frontiers in Artificial Intelligence planning reinforcement learning model-based reinforcement learning framework overview synthesis |
| title | A Unifying Framework for Reinforcement Learning and Planning |
| title_full | A Unifying Framework for Reinforcement Learning and Planning |
| title_fullStr | A Unifying Framework for Reinforcement Learning and Planning |
| title_full_unstemmed | A Unifying Framework for Reinforcement Learning and Planning |
| title_short | A Unifying Framework for Reinforcement Learning and Planning |
| title_sort | unifying framework for reinforcement learning and planning |
| topic | planning reinforcement learning model-based reinforcement learning framework overview synthesis |
| url | https://www.frontiersin.org/articles/10.3389/frai.2022.908353/full |
| work_keys_str_mv | AT thomasmmoerland aunifyingframeworkforreinforcementlearningandplanning AT joostbroekens aunifyingframeworkforreinforcementlearningandplanning AT askeplaat aunifyingframeworkforreinforcementlearningandplanning AT catholijnmjonker aunifyingframeworkforreinforcementlearningandplanning AT catholijnmjonker aunifyingframeworkforreinforcementlearningandplanning AT thomasmmoerland unifyingframeworkforreinforcementlearningandplanning AT joostbroekens unifyingframeworkforreinforcementlearningandplanning AT askeplaat unifyingframeworkforreinforcementlearningandplanning AT catholijnmjonker unifyingframeworkforreinforcementlearningandplanning AT catholijnmjonker unifyingframeworkforreinforcementlearningandplanning |