Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints
IntroductionReinforcement learning has been widely used in robot motion planning. However, for multi-step complex tasks of dual-arm robots, the trajectory planning method based on reinforcement learning still has some problems, such as ample exploration space, long training time, and uncontrollable...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-02-01
|
| Series: | Frontiers in Neurorobotics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnbot.2024.1362359/full |
| _version_ | 1827345391485452288 |
|---|---|
| author | Keyao Liang Fusheng Zha Wei Guo Shengkai Liu Pengfei Wang Lining Sun |
| author_facet | Keyao Liang Fusheng Zha Wei Guo Shengkai Liu Pengfei Wang Lining Sun |
| author_sort | Keyao Liang |
| collection | DOAJ |
| description | IntroductionReinforcement learning has been widely used in robot motion planning. However, for multi-step complex tasks of dual-arm robots, the trajectory planning method based on reinforcement learning still has some problems, such as ample exploration space, long training time, and uncontrollable training process. Based on the dual-agent depth deterministic strategy gradient (DADDPG) algorithm, this study proposes a motion planning framework constrained by the human joint angle, simultaneously realizing the humanization of learning content and learning style. It quickly plans the coordinated trajectory of dual-arm for complex multi-step tasks.MethodsThe proposed framework mainly includes two parts: one is the modeling of human joint angle constraints. The joint angle is calculated from the human arm motion data measured by the inertial measurement unit (IMU) by establishing a human-robot dual-arm kinematic mapping model. Then, the joint angle range constraints are extracted from multiple groups of demonstration data and expressed as inequalities. Second, the segmented reward function is designed. The human joint angle constraint guides the exploratory learning process of the reinforcement learning method in the form of step reward. Therefore, the exploration space is reduced, the training speed is accelerated, and the learning process is controllable to a certain extent.Results and discussionThe effectiveness of the framework was verified in the gym simulation environment of the Baxter robot's reach-grasp-align task. The results show that in this framework, human experience knowledge has a significant impact on the guidance of learning, and this method can more quickly plan the coordinated trajectory of dual-arm for multi-step tasks. |
| first_indexed | 2024-03-07T23:06:59Z |
| format | Article |
| id | doaj.art-559eaf12825045728299bd5de569bae1 |
| institution | Directory Open Access Journal |
| issn | 1662-5218 |
| language | English |
| last_indexed | 2024-03-07T23:06:59Z |
| publishDate | 2024-02-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurorobotics |
| spelling | doaj.art-559eaf12825045728299bd5de569bae12024-02-22T04:43:58ZengFrontiers Media S.A.Frontiers in Neurorobotics1662-52182024-02-011810.3389/fnbot.2024.13623591362359Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraintsKeyao LiangFusheng ZhaWei GuoShengkai LiuPengfei WangLining SunIntroductionReinforcement learning has been widely used in robot motion planning. However, for multi-step complex tasks of dual-arm robots, the trajectory planning method based on reinforcement learning still has some problems, such as ample exploration space, long training time, and uncontrollable training process. Based on the dual-agent depth deterministic strategy gradient (DADDPG) algorithm, this study proposes a motion planning framework constrained by the human joint angle, simultaneously realizing the humanization of learning content and learning style. It quickly plans the coordinated trajectory of dual-arm for complex multi-step tasks.MethodsThe proposed framework mainly includes two parts: one is the modeling of human joint angle constraints. The joint angle is calculated from the human arm motion data measured by the inertial measurement unit (IMU) by establishing a human-robot dual-arm kinematic mapping model. Then, the joint angle range constraints are extracted from multiple groups of demonstration data and expressed as inequalities. Second, the segmented reward function is designed. The human joint angle constraint guides the exploratory learning process of the reinforcement learning method in the form of step reward. Therefore, the exploration space is reduced, the training speed is accelerated, and the learning process is controllable to a certain extent.Results and discussionThe effectiveness of the framework was verified in the gym simulation environment of the Baxter robot's reach-grasp-align task. The results show that in this framework, human experience knowledge has a significant impact on the guidance of learning, and this method can more quickly plan the coordinated trajectory of dual-arm for multi-step tasks.https://www.frontiersin.org/articles/10.3389/fnbot.2024.1362359/fulltrajectory planningreinforcement learningdual-agent depth deterministic strategy gradienthuman experience constrains guidancemotion parameter mapping |
| spellingShingle | Keyao Liang Fusheng Zha Wei Guo Shengkai Liu Pengfei Wang Lining Sun Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints Frontiers in Neurorobotics trajectory planning reinforcement learning dual-agent depth deterministic strategy gradient human experience constrains guidance motion parameter mapping |
| title | Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints |
| title_full | Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints |
| title_fullStr | Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints |
| title_full_unstemmed | Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints |
| title_short | Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints |
| title_sort | motion planning framework based on dual agent ddpg method for dual arm robots guided by human joint angle constraints |
| topic | trajectory planning reinforcement learning dual-agent depth deterministic strategy gradient human experience constrains guidance motion parameter mapping |
| url | https://www.frontiersin.org/articles/10.3389/fnbot.2024.1362359/full |
| work_keys_str_mv | AT keyaoliang motionplanningframeworkbasedondualagentddpgmethodfordualarmrobotsguidedbyhumanjointangleconstraints AT fushengzha motionplanningframeworkbasedondualagentddpgmethodfordualarmrobotsguidedbyhumanjointangleconstraints AT weiguo motionplanningframeworkbasedondualagentddpgmethodfordualarmrobotsguidedbyhumanjointangleconstraints AT shengkailiu motionplanningframeworkbasedondualagentddpgmethodfordualarmrobotsguidedbyhumanjointangleconstraints AT pengfeiwang motionplanningframeworkbasedondualagentddpgmethodfordualarmrobotsguidedbyhumanjointangleconstraints AT liningsun motionplanningframeworkbasedondualagentddpgmethodfordualarmrobotsguidedbyhumanjointangleconstraints |