A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning
Collaborative control of a dual-arm robot refers to collision avoidance and working together to accomplish a task. To prevent the collision of two arms, the control strategy of a robot arm needs to avoid competition and to cooperate with the other one during motion planning. In this paper, a dual-ar...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-02-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/4/1816 |
| _version_ | 1797396113500667904 |
|---|---|
| author | Luyu Liu Qianyuan Liu Yong Song Bao Pang Xianfeng Yuan Qingyang Xu |
| author_facet | Luyu Liu Qianyuan Liu Yong Song Bao Pang Xianfeng Yuan Qingyang Xu |
| author_sort | Luyu Liu |
| collection | DOAJ |
| description | Collaborative control of a dual-arm robot refers to collision avoidance and working together to accomplish a task. To prevent the collision of two arms, the control strategy of a robot arm needs to avoid competition and to cooperate with the other one during motion planning. In this paper, a dual-arm deep deterministic policy gradient (DADDPG) algorithm is proposed based on deep reinforcement learning of multi-agent cooperation. Firstly, the construction method of a replay buffer in a hindsight experience replay algorithm is introduced. The modeling and training method of the multi-agent deep deterministic policy gradient algorithm is explained. Secondly, a control strategy is assigned to each robotic arm. The arms share their observations and actions. The dual-arm robot is trained based on a mechanism of “rewarding cooperation and punishing competition”. Finally, the effectiveness of the algorithm is verified in the Reach, Push, and Pick up simulation environment built in this study. The experiment results show that the robot trained by the DADDPG algorithm can achieve cooperative tasks. The algorithm can make the robots explore the action space autonomously and reduce the level of competition with each other. The collaborative robots have better adaptability to coordination tasks. |
| first_indexed | 2024-03-09T00:45:37Z |
| format | Article |
| id | doaj.art-6151aa04afc6466bb1e9d5db08ddcf9c |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T00:45:37Z |
| publishDate | 2021-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-6151aa04afc6466bb1e9d5db08ddcf9c2023-12-11T17:30:27ZengMDPI AGApplied Sciences2076-34172021-02-01114181610.3390/app11041816A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement LearningLuyu Liu0Qianyuan Liu1Yong Song2Bao Pang3Xianfeng Yuan4Qingyang Xu5School of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264200, ChinaSchool of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264200, ChinaSchool of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264200, ChinaSchool of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264200, ChinaSchool of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264200, ChinaSchool of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264200, ChinaCollaborative control of a dual-arm robot refers to collision avoidance and working together to accomplish a task. To prevent the collision of two arms, the control strategy of a robot arm needs to avoid competition and to cooperate with the other one during motion planning. In this paper, a dual-arm deep deterministic policy gradient (DADDPG) algorithm is proposed based on deep reinforcement learning of multi-agent cooperation. Firstly, the construction method of a replay buffer in a hindsight experience replay algorithm is introduced. The modeling and training method of the multi-agent deep deterministic policy gradient algorithm is explained. Secondly, a control strategy is assigned to each robotic arm. The arms share their observations and actions. The dual-arm robot is trained based on a mechanism of “rewarding cooperation and punishing competition”. Finally, the effectiveness of the algorithm is verified in the Reach, Push, and Pick up simulation environment built in this study. The experiment results show that the robot trained by the DADDPG algorithm can achieve cooperative tasks. The algorithm can make the robots explore the action space autonomously and reduce the level of competition with each other. The collaborative robots have better adaptability to coordination tasks.https://www.mdpi.com/2076-3417/11/4/1816deep reinforcement learningmulti-agent collaborationdual-arm robotcoordinated manipulation |
| spellingShingle | Luyu Liu Qianyuan Liu Yong Song Bao Pang Xianfeng Yuan Qingyang Xu A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning Applied Sciences deep reinforcement learning multi-agent collaboration dual-arm robot coordinated manipulation |
| title | A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning |
| title_full | A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning |
| title_fullStr | A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning |
| title_full_unstemmed | A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning |
| title_short | A Collaborative Control Method of Dual-Arm Robots Based on Deep Reinforcement Learning |
| title_sort | collaborative control method of dual arm robots based on deep reinforcement learning |
| topic | deep reinforcement learning multi-agent collaboration dual-arm robot coordinated manipulation |
| url | https://www.mdpi.com/2076-3417/11/4/1816 |
| work_keys_str_mv | AT luyuliu acollaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT qianyuanliu acollaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT yongsong acollaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT baopang acollaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT xianfengyuan acollaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT qingyangxu acollaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT luyuliu collaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT qianyuanliu collaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT yongsong collaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT baopang collaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT xianfengyuan collaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning AT qingyangxu collaborativecontrolmethodofdualarmrobotsbasedondeepreinforcementlearning |