Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured Environment
When a lunar assisted robot helps an astronaut turn over or transports the astronaut from the ground, the trajectory of the robot’s dual arms should be automatically planned according to the unstructured environment on the lunar surface. In this paper, a dual-arm control strategy model of a lunar as...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Aerospace |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2226-4310/9/6/315 |
| _version_ | 1797491047373209600 |
|---|---|
| author | Weiyan Ren Dapeng Han Zhaokui Wang |
| author_facet | Weiyan Ren Dapeng Han Zhaokui Wang |
| author_sort | Weiyan Ren |
| collection | DOAJ |
| description | When a lunar assisted robot helps an astronaut turn over or transports the astronaut from the ground, the trajectory of the robot’s dual arms should be automatically planned according to the unstructured environment on the lunar surface. In this paper, a dual-arm control strategy model of a lunar assisted robot based on hierarchical reinforcement learning is proposed, and the trajectory planning problem is modeled as a two-layer Markov decision process. In the training process, a reward function design method based on the idea of the artificial potential field method is proposed, and the reward information is fed back in a dense reward method, which significantly reduces the invalid exploration space and improves the learning efficiency. Large-scale tests are carried out in both simulated and physical environments, and the results demonstrate the effectiveness of the method proposed in this paper. This research is of great significance in respect of human–robot interaction, environmental interaction, and intelligent control of robots. |
| first_indexed | 2024-03-10T00:41:49Z |
| format | Article |
| id | doaj.art-059cbb95cb7e492bbfd79f0db7bef09b |
| institution | Directory Open Access Journal |
| issn | 2226-4310 |
| language | English |
| last_indexed | 2024-03-10T00:41:49Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Aerospace |
| spelling | doaj.art-059cbb95cb7e492bbfd79f0db7bef09b2023-11-23T15:05:38ZengMDPI AGAerospace2226-43102022-06-019631510.3390/aerospace9060315Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured EnvironmentWeiyan Ren0Dapeng Han1Zhaokui Wang2School of Aerospace Engineering, Tsinghua University, Beijing 100084, ChinaSchool of Aerospace Engineering, Tsinghua University, Beijing 100084, ChinaSchool of Aerospace Engineering, Tsinghua University, Beijing 100084, ChinaWhen a lunar assisted robot helps an astronaut turn over or transports the astronaut from the ground, the trajectory of the robot’s dual arms should be automatically planned according to the unstructured environment on the lunar surface. In this paper, a dual-arm control strategy model of a lunar assisted robot based on hierarchical reinforcement learning is proposed, and the trajectory planning problem is modeled as a two-layer Markov decision process. In the training process, a reward function design method based on the idea of the artificial potential field method is proposed, and the reward information is fed back in a dense reward method, which significantly reduces the invalid exploration space and improves the learning efficiency. Large-scale tests are carried out in both simulated and physical environments, and the results demonstrate the effectiveness of the method proposed in this paper. This research is of great significance in respect of human–robot interaction, environmental interaction, and intelligent control of robots.https://www.mdpi.com/2226-4310/9/6/315lunar assisted robothierarchical reinforcement learningdual-arm controlreward functions |
| spellingShingle | Weiyan Ren Dapeng Han Zhaokui Wang Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured Environment Aerospace lunar assisted robot hierarchical reinforcement learning dual-arm control reward functions |
| title | Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured Environment |
| title_full | Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured Environment |
| title_fullStr | Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured Environment |
| title_full_unstemmed | Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured Environment |
| title_short | Research on Dual-Arm Control of Lunar Assisted Robot Based on Hierarchical Reinforcement Learning under Unstructured Environment |
| title_sort | research on dual arm control of lunar assisted robot based on hierarchical reinforcement learning under unstructured environment |
| topic | lunar assisted robot hierarchical reinforcement learning dual-arm control reward functions |
| url | https://www.mdpi.com/2226-4310/9/6/315 |
| work_keys_str_mv | AT weiyanren researchondualarmcontroloflunarassistedrobotbasedonhierarchicalreinforcementlearningunderunstructuredenvironment AT dapenghan researchondualarmcontroloflunarassistedrobotbasedonhierarchicalreinforcementlearningunderunstructuredenvironment AT zhaokuiwang researchondualarmcontroloflunarassistedrobotbasedonhierarchicalreinforcementlearningunderunstructuredenvironment |