Learning 3D Bipedal Walking with Planned Footsteps and Fourier Series Periodic Gait Planning
Reinforcement learning provides a general framework for achieving autonomy and diversity in traditional robot motion control. Robots must walk dynamically to adapt to different ground environments in complex environments. To achieve walking ability similar to that of humans, robots must be able to p...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/4/1873 |
| _version_ | 1827755624449966080 |
|---|---|
| author | Song Wang Songhao Piao Xiaokun Leng Zhicheng He |
| author_facet | Song Wang Songhao Piao Xiaokun Leng Zhicheng He |
| author_sort | Song Wang |
| collection | DOAJ |
| description | Reinforcement learning provides a general framework for achieving autonomy and diversity in traditional robot motion control. Robots must walk dynamically to adapt to different ground environments in complex environments. To achieve walking ability similar to that of humans, robots must be able to perceive, understand and interact with the surrounding environment. In 3D environments, walking like humans on rugged terrain is a challenging task because it requires complex world model generation, motion planning and control algorithms and their integration. So, the learning of high-dimensional complex motions is still a hot topic in research. This paper proposes a deep reinforcement learning-based footstep tracking method, which tracks the robot’s footstep position by adding periodic and symmetrical information of bipedal walking to the reward function. The robot can achieve robot obstacle avoidance and omnidirectional walking, turning, standing and climbing stairs in complex environments. Experimental results show that reinforcement learning can be combined with real-time robot footstep planning, avoiding the learning of path-planning information in the model training process, so as to avoid the model learning unnecessary knowledge and thereby accelerate the training process. |
| first_indexed | 2024-03-11T08:11:38Z |
| format | Article |
| id | doaj.art-28286e19f6d845ec91c4f4afdc227fcf |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T08:11:38Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-28286e19f6d845ec91c4f4afdc227fcf2023-11-16T23:07:16ZengMDPI AGSensors1424-82202023-02-01234187310.3390/s23041873Learning 3D Bipedal Walking with Planned Footsteps and Fourier Series Periodic Gait PlanningSong Wang0Songhao Piao1Xiaokun Leng2Zhicheng He3School of Computer Science, Harbin Institute of Technology, Harbin 150009, ChinaSchool of Computer Science, Harbin Institute of Technology, Harbin 150009, ChinaSchool of Computer Science, Harbin Institute of Technology, Harbin 150009, ChinaSchool of Computer Science, Harbin Institute of Technology, Harbin 150009, ChinaReinforcement learning provides a general framework for achieving autonomy and diversity in traditional robot motion control. Robots must walk dynamically to adapt to different ground environments in complex environments. To achieve walking ability similar to that of humans, robots must be able to perceive, understand and interact with the surrounding environment. In 3D environments, walking like humans on rugged terrain is a challenging task because it requires complex world model generation, motion planning and control algorithms and their integration. So, the learning of high-dimensional complex motions is still a hot topic in research. This paper proposes a deep reinforcement learning-based footstep tracking method, which tracks the robot’s footstep position by adding periodic and symmetrical information of bipedal walking to the reward function. The robot can achieve robot obstacle avoidance and omnidirectional walking, turning, standing and climbing stairs in complex environments. Experimental results show that reinforcement learning can be combined with real-time robot footstep planning, avoiding the learning of path-planning information in the model training process, so as to avoid the model learning unnecessary knowledge and thereby accelerate the training process.https://www.mdpi.com/1424-8220/23/4/1873humanoidfootstep planningreinforcement learninggait phase |
| spellingShingle | Song Wang Songhao Piao Xiaokun Leng Zhicheng He Learning 3D Bipedal Walking with Planned Footsteps and Fourier Series Periodic Gait Planning Sensors humanoid footstep planning reinforcement learning gait phase |
| title | Learning 3D Bipedal Walking with Planned Footsteps and Fourier Series Periodic Gait Planning |
| title_full | Learning 3D Bipedal Walking with Planned Footsteps and Fourier Series Periodic Gait Planning |
| title_fullStr | Learning 3D Bipedal Walking with Planned Footsteps and Fourier Series Periodic Gait Planning |
| title_full_unstemmed | Learning 3D Bipedal Walking with Planned Footsteps and Fourier Series Periodic Gait Planning |
| title_short | Learning 3D Bipedal Walking with Planned Footsteps and Fourier Series Periodic Gait Planning |
| title_sort | learning 3d bipedal walking with planned footsteps and fourier series periodic gait planning |
| topic | humanoid footstep planning reinforcement learning gait phase |
| url | https://www.mdpi.com/1424-8220/23/4/1873 |
| work_keys_str_mv | AT songwang learning3dbipedalwalkingwithplannedfootstepsandfourierseriesperiodicgaitplanning AT songhaopiao learning3dbipedalwalkingwithplannedfootstepsandfourierseriesperiodicgaitplanning AT xiaokunleng learning3dbipedalwalkingwithplannedfootstepsandfourierseriesperiodicgaitplanning AT zhichenghe learning3dbipedalwalkingwithplannedfootstepsandfourierseriesperiodicgaitplanning |