Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion Tasks
Domain randomization in the context of Reinforcement learning (RL) involves training RL agents with randomized environmental properties or parameters to improve the generalization capabilities of the resulting agents. Although domain randomization has been favorably studied in the literature, it has...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/23/4744 |
| _version_ | 1797399819554127872 |
|---|---|
| author | Oladayo S. Ajani Sung-ho Hur Rammohan Mallipeddi |
| author_facet | Oladayo S. Ajani Sung-ho Hur Rammohan Mallipeddi |
| author_sort | Oladayo S. Ajani |
| collection | DOAJ |
| description | Domain randomization in the context of Reinforcement learning (RL) involves training RL agents with randomized environmental properties or parameters to improve the generalization capabilities of the resulting agents. Although domain randomization has been favorably studied in the literature, it has been studied in terms of varying the operational characters of the associated systems or physical dynamics rather than their environmental characteristics. This is counter-intuitive as it is unrealistic to alter the mechanical dynamics of a system in operation. Furthermore, most works were based on cherry-picked environments within different classes of RL tasks. Therefore, in this work, we investigated domain randomization by varying only the properties or parameters of the environment rather than varying the mechanical dynamics of the featured systems. Furthermore, the analysis conducted was based on all six RL locomotion tasks. In terms of training the RL agents, we employed two proven RL algorithms (SAC and TD3) and evaluated the generalization capabilities of the resulting agents on several train–test scenarios that involve both in-distribution and out-distribution evaluations as well as scenarios applicable in the real world. The results demonstrate that, although domain randomization favors generalization, some tasks only require randomization from low-dimensional distributions while others require randomization from high-dimensional randomization. Hence the question of what level of randomization is optimal for any given task becomes very important. |
| first_indexed | 2024-03-09T01:46:35Z |
| format | Article |
| id | doaj.art-0c2bb5c2116f41ba9fd39f92fb1d0abc |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-09T01:46:35Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-0c2bb5c2116f41ba9fd39f92fb1d0abc2023-12-08T15:21:38ZengMDPI AGMathematics2227-73902023-11-011123474410.3390/math11234744Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion TasksOladayo S. Ajani0Sung-ho Hur1Rammohan Mallipeddi2School of Electronics Engineering, Kyungpook National University, Daegu 37224, Republic of KoreaSchool of Electronics Engineering, Kyungpook National University, Daegu 37224, Republic of KoreaSchool of Electronics Engineering, Kyungpook National University, Daegu 37224, Republic of KoreaDomain randomization in the context of Reinforcement learning (RL) involves training RL agents with randomized environmental properties or parameters to improve the generalization capabilities of the resulting agents. Although domain randomization has been favorably studied in the literature, it has been studied in terms of varying the operational characters of the associated systems or physical dynamics rather than their environmental characteristics. This is counter-intuitive as it is unrealistic to alter the mechanical dynamics of a system in operation. Furthermore, most works were based on cherry-picked environments within different classes of RL tasks. Therefore, in this work, we investigated domain randomization by varying only the properties or parameters of the environment rather than varying the mechanical dynamics of the featured systems. Furthermore, the analysis conducted was based on all six RL locomotion tasks. In terms of training the RL agents, we employed two proven RL algorithms (SAC and TD3) and evaluated the generalization capabilities of the resulting agents on several train–test scenarios that involve both in-distribution and out-distribution evaluations as well as scenarios applicable in the real world. The results demonstrate that, although domain randomization favors generalization, some tasks only require randomization from low-dimensional distributions while others require randomization from high-dimensional randomization. Hence the question of what level of randomization is optimal for any given task becomes very important.https://www.mdpi.com/2227-7390/11/23/4744generalizationdeep reinforcement learningdynamic environmentslocomotiondomain randomization |
| spellingShingle | Oladayo S. Ajani Sung-ho Hur Rammohan Mallipeddi Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion Tasks Mathematics generalization deep reinforcement learning dynamic environments locomotion domain randomization |
| title | Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion Tasks |
| title_full | Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion Tasks |
| title_fullStr | Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion Tasks |
| title_full_unstemmed | Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion Tasks |
| title_short | Evaluating Domain Randomization in Deep Reinforcement Learning Locomotion Tasks |
| title_sort | evaluating domain randomization in deep reinforcement learning locomotion tasks |
| topic | generalization deep reinforcement learning dynamic environments locomotion domain randomization |
| url | https://www.mdpi.com/2227-7390/11/23/4744 |
| work_keys_str_mv | AT oladayosajani evaluatingdomainrandomizationindeepreinforcementlearninglocomotiontasks AT sunghohur evaluatingdomainrandomizationindeepreinforcementlearninglocomotiontasks AT rammohanmallipeddi evaluatingdomainrandomizationindeepreinforcementlearninglocomotiontasks |