Reinforcement Learning with Dynamic Movement Primitives for Obstacle Avoidance
Dynamic movement primitives (DMPs) are a robust framework for movement generation from demonstrations. This framework can be extended by adding a perturbing term to achieve obstacle avoidance without sacrificing stability. The additional term is usually constructed based on potential functions. Alth...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-11-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/23/11184 |
| _version_ | 1797508069510348800 |
|---|---|
| author | Ang Li Zhenze Liu Wenrui Wang Mingchao Zhu Yanhui Li Qi Huo Ming Dai |
| author_facet | Ang Li Zhenze Liu Wenrui Wang Mingchao Zhu Yanhui Li Qi Huo Ming Dai |
| author_sort | Ang Li |
| collection | DOAJ |
| description | Dynamic movement primitives (DMPs) are a robust framework for movement generation from demonstrations. This framework can be extended by adding a perturbing term to achieve obstacle avoidance without sacrificing stability. The additional term is usually constructed based on potential functions. Although different potentials are adopted to improve the performance of obstacle avoidance, the profiles of potentials are rarely incorporated into reinforcement learning (RL) framework. In this contribution, we present a RL based method to learn not only the profiles of potentials but also the shape parameters of a motion. The algorithm employed is PI2 (Policy Improvement with Path Integrals), a model-free, sampling-based learning method. By using the PI2, the profiles of potentials and the parameters of the DMPs are learned simultaneously; therefore, we can optimize obstacle avoidance while completing specified tasks. We validate the presented method in simulations and with a redundant robot arm in experiments. |
| first_indexed | 2024-03-10T04:58:02Z |
| format | Article |
| id | doaj.art-4821f3a499de4af392173b06decd54ef |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T04:58:02Z |
| publishDate | 2021-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-4821f3a499de4af392173b06decd54ef2023-11-23T02:03:51ZengMDPI AGApplied Sciences2076-34172021-11-0111231118410.3390/app112311184Reinforcement Learning with Dynamic Movement Primitives for Obstacle AvoidanceAng Li0Zhenze Liu1Wenrui Wang2Mingchao Zhu3Yanhui Li4Qi Huo5Ming Dai6Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaCollege of Communication Engineering, Jilin University, Changchun 130025, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaChangchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, ChinaDynamic movement primitives (DMPs) are a robust framework for movement generation from demonstrations. This framework can be extended by adding a perturbing term to achieve obstacle avoidance without sacrificing stability. The additional term is usually constructed based on potential functions. Although different potentials are adopted to improve the performance of obstacle avoidance, the profiles of potentials are rarely incorporated into reinforcement learning (RL) framework. In this contribution, we present a RL based method to learn not only the profiles of potentials but also the shape parameters of a motion. The algorithm employed is PI2 (Policy Improvement with Path Integrals), a model-free, sampling-based learning method. By using the PI2, the profiles of potentials and the parameters of the DMPs are learned simultaneously; therefore, we can optimize obstacle avoidance while completing specified tasks. We validate the presented method in simulations and with a redundant robot arm in experiments.https://www.mdpi.com/2076-3417/11/23/11184obstacle avoidanceDynamic Movement Primitivesreinforcement learningPI2 (policy improvement with path integrals) |
| spellingShingle | Ang Li Zhenze Liu Wenrui Wang Mingchao Zhu Yanhui Li Qi Huo Ming Dai Reinforcement Learning with Dynamic Movement Primitives for Obstacle Avoidance Applied Sciences obstacle avoidance Dynamic Movement Primitives reinforcement learning PI2 (policy improvement with path integrals) |
| title | Reinforcement Learning with Dynamic Movement Primitives for Obstacle Avoidance |
| title_full | Reinforcement Learning with Dynamic Movement Primitives for Obstacle Avoidance |
| title_fullStr | Reinforcement Learning with Dynamic Movement Primitives for Obstacle Avoidance |
| title_full_unstemmed | Reinforcement Learning with Dynamic Movement Primitives for Obstacle Avoidance |
| title_short | Reinforcement Learning with Dynamic Movement Primitives for Obstacle Avoidance |
| title_sort | reinforcement learning with dynamic movement primitives for obstacle avoidance |
| topic | obstacle avoidance Dynamic Movement Primitives reinforcement learning PI2 (policy improvement with path integrals) |
| url | https://www.mdpi.com/2076-3417/11/23/11184 |
| work_keys_str_mv | AT angli reinforcementlearningwithdynamicmovementprimitivesforobstacleavoidance AT zhenzeliu reinforcementlearningwithdynamicmovementprimitivesforobstacleavoidance AT wenruiwang reinforcementlearningwithdynamicmovementprimitivesforobstacleavoidance AT mingchaozhu reinforcementlearningwithdynamicmovementprimitivesforobstacleavoidance AT yanhuili reinforcementlearningwithdynamicmovementprimitivesforobstacleavoidance AT qihuo reinforcementlearningwithdynamicmovementprimitivesforobstacleavoidance AT mingdai reinforcementlearningwithdynamicmovementprimitivesforobstacleavoidance |