Real-time trajectory adaptation for quadrupedal locomotion using deep reinforcement learning
We present a control architecture for real-time adaptation and tracking of trajectories generated using a terrain-aware trajectory optimization solver. This approach enables us to circumvent the computationally exhaustive task of online trajectory optimization, and further introduces a control solut...
| Main Authors: | , , , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
IEEE
2021
|
| _version_ | 1797085998244429824 |
|---|---|
| author | Gangapurwala, S Geisert, M Orsolino, R Fallon, M Havoutis, I |
| author_facet | Gangapurwala, S Geisert, M Orsolino, R Fallon, M Havoutis, I |
| author_sort | Gangapurwala, S |
| collection | OXFORD |
| description | We present a control architecture for real-time
adaptation and tracking of trajectories generated using a
terrain-aware trajectory optimization solver. This approach
enables us to circumvent the computationally exhaustive task
of online trajectory optimization, and further introduces a
control solution robust to systems modeled with approximated
dynamics. We train a policy using deep reinforcement learning
(RL) to introduce additive deviations to a reference trajectory
in order to generate a feedback-based trajectory tracking
system for a quadrupedal robot. We train this policy across
a multitude of simulated terrains and ensure its generality
by introducing training methods that avoid overfitting and
convergence towards local optima. Additionally, in order to
capture terrain information, we include a latent representation of the height maps in the observation space of the RL
environment as a form of exteroceptive feedback. We test the
performance of our trained policy by tracking the corrected set
points using a model-based whole-body controller and compare
it with the tracking behavior obtained without the corrective
feedback in several simulation environments. We also show
successful transfer of our training approach to the real physical
system and further present cogent arguments in support of our
framework. |
| first_indexed | 2024-03-07T02:15:45Z |
| format | Conference item |
| id | oxford-uuid:a22e502f-b7ef-4c6e-9426-31812e180cd8 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T02:15:45Z |
| publishDate | 2021 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:a22e502f-b7ef-4c6e-9426-31812e180cd82022-03-27T02:18:27ZReal-time trajectory adaptation for quadrupedal locomotion using deep reinforcement learningConference itemhttp://purl.org/coar/resource_type/c_5794uuid:a22e502f-b7ef-4c6e-9426-31812e180cd8EnglishSymplectic ElementsIEEE2021Gangapurwala, SGeisert, MOrsolino, RFallon, MHavoutis, IWe present a control architecture for real-time adaptation and tracking of trajectories generated using a terrain-aware trajectory optimization solver. This approach enables us to circumvent the computationally exhaustive task of online trajectory optimization, and further introduces a control solution robust to systems modeled with approximated dynamics. We train a policy using deep reinforcement learning (RL) to introduce additive deviations to a reference trajectory in order to generate a feedback-based trajectory tracking system for a quadrupedal robot. We train this policy across a multitude of simulated terrains and ensure its generality by introducing training methods that avoid overfitting and convergence towards local optima. Additionally, in order to capture terrain information, we include a latent representation of the height maps in the observation space of the RL environment as a form of exteroceptive feedback. We test the performance of our trained policy by tracking the corrected set points using a model-based whole-body controller and compare it with the tracking behavior obtained without the corrective feedback in several simulation environments. We also show successful transfer of our training approach to the real physical system and further present cogent arguments in support of our framework. |
| spellingShingle | Gangapurwala, S Geisert, M Orsolino, R Fallon, M Havoutis, I Real-time trajectory adaptation for quadrupedal locomotion using deep reinforcement learning |
| title | Real-time trajectory adaptation for quadrupedal locomotion using deep reinforcement learning |
| title_full | Real-time trajectory adaptation for quadrupedal locomotion using deep reinforcement learning |
| title_fullStr | Real-time trajectory adaptation for quadrupedal locomotion using deep reinforcement learning |
| title_full_unstemmed | Real-time trajectory adaptation for quadrupedal locomotion using deep reinforcement learning |
| title_short | Real-time trajectory adaptation for quadrupedal locomotion using deep reinforcement learning |
| title_sort | real time trajectory adaptation for quadrupedal locomotion using deep reinforcement learning |
| work_keys_str_mv | AT gangapurwalas realtimetrajectoryadaptationforquadrupedallocomotionusingdeepreinforcementlearning AT geisertm realtimetrajectoryadaptationforquadrupedallocomotionusingdeepreinforcementlearning AT orsolinor realtimetrajectoryadaptationforquadrupedallocomotionusingdeepreinforcementlearning AT fallonm realtimetrajectoryadaptationforquadrupedallocomotionusingdeepreinforcementlearning AT havoutisi realtimetrajectoryadaptationforquadrupedallocomotionusingdeepreinforcementlearning |