Exploring Parameter Space in Reinforcement Learning
This paper discusses parameter-based exploration methods for reinforcement learning. Parameter-based methods perturb parameters of a general function approximator directly, rather than adding noise to the resulting actions. Parameter-based exploration unifies reinforcement learning and black-box opt...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
De Gruyter
2010-03-01
|
| Series: | Paladyn |
| Subjects: | |
| Online Access: | https://doi.org/10.2478/s13230-010-0002-4 |
| _version_ | 1797668818136334336 |
|---|---|
| author | Rückstieß Thomas Sehnke Frank Schaul Tom Wierstra Daan Sun Yi Schmidhuber Jürgen |
| author_facet | Rückstieß Thomas Sehnke Frank Schaul Tom Wierstra Daan Sun Yi Schmidhuber Jürgen |
| author_sort | Rückstieß Thomas |
| collection | DOAJ |
| description | This paper discusses parameter-based exploration methods for reinforcement learning. Parameter-based methods perturb parameters of a general function approximator directly, rather than adding noise to the resulting actions. Parameter-based exploration unifies reinforcement learning and black-box optimization, and has several advantages over action perturbation. We review two recent parameter-exploring algorithms: Natural Evolution Strategies and Policy Gradients with Parameter-Based Exploration. Both outperform state-of-the-art algorithms in several complex high-dimensional tasks commonly found in robot control. Furthermore, we describe how a novel exploration method, State-Dependent Exploration, can modify existing algorithms to mimic exploration in parameter space. |
| first_indexed | 2024-03-11T20:35:00Z |
| format | Article |
| id | doaj.art-1d7d7f4ee4c14115b57878db43f50fab |
| institution | Directory Open Access Journal |
| issn | 2081-4836 |
| language | English |
| last_indexed | 2024-03-11T20:35:00Z |
| publishDate | 2010-03-01 |
| publisher | De Gruyter |
| record_format | Article |
| series | Paladyn |
| spelling | doaj.art-1d7d7f4ee4c14115b57878db43f50fab2023-10-02T07:10:43ZengDe GruyterPaladyn2081-48362010-03-0111142410.2478/s13230-010-0002-4Exploring Parameter Space in Reinforcement LearningRückstieß Thomas0Sehnke Frank1Schaul Tom2Wierstra Daan3Sun Yi4Schmidhuber Jürgen5 Technische Universität München, Institut für Informatik VI, Boltzmannstr. 3, 85748 Garching, Germany Technische Universität München, Institut für Informatik VI, Boltzmannstr. 3, 85748 Garching, Germany Dalle Molle Institute for Artificial Intelligence (IDSIA), Galleria 2, 6928 Manno-Lugano, Switzerland Dalle Molle Institute for Artificial Intelligence (IDSIA), Galleria 2, 6928 Manno-Lugano, Switzerland Dalle Molle Institute for Artificial Intelligence (IDSIA), Galleria 2, 6928 Manno-Lugano, Switzerland Dalle Molle Institute for Artificial Intelligence (IDSIA), Galleria 2, 6928 Manno-Lugano, SwitzerlandThis paper discusses parameter-based exploration methods for reinforcement learning. Parameter-based methods perturb parameters of a general function approximator directly, rather than adding noise to the resulting actions. Parameter-based exploration unifies reinforcement learning and black-box optimization, and has several advantages over action perturbation. We review two recent parameter-exploring algorithms: Natural Evolution Strategies and Policy Gradients with Parameter-Based Exploration. Both outperform state-of-the-art algorithms in several complex high-dimensional tasks commonly found in robot control. Furthermore, we describe how a novel exploration method, State-Dependent Exploration, can modify existing algorithms to mimic exploration in parameter space.https://doi.org/10.2478/s13230-010-0002-4reinforcement learningoptimizationexplorationpolicy gradients |
| spellingShingle | Rückstieß Thomas Sehnke Frank Schaul Tom Wierstra Daan Sun Yi Schmidhuber Jürgen Exploring Parameter Space in Reinforcement Learning Paladyn reinforcement learning optimization exploration policy gradients |
| title | Exploring Parameter Space in Reinforcement Learning |
| title_full | Exploring Parameter Space in Reinforcement Learning |
| title_fullStr | Exploring Parameter Space in Reinforcement Learning |
| title_full_unstemmed | Exploring Parameter Space in Reinforcement Learning |
| title_short | Exploring Parameter Space in Reinforcement Learning |
| title_sort | exploring parameter space in reinforcement learning |
| topic | reinforcement learning optimization exploration policy gradients |
| url | https://doi.org/10.2478/s13230-010-0002-4 |
| work_keys_str_mv | AT ruckstießthomas exploringparameterspaceinreinforcementlearning AT sehnkefrank exploringparameterspaceinreinforcementlearning AT schaultom exploringparameterspaceinreinforcementlearning AT wierstradaan exploringparameterspaceinreinforcementlearning AT sunyi exploringparameterspaceinreinforcementlearning AT schmidhuberjurgen exploringparameterspaceinreinforcementlearning |