Characterization of continuum robot arms under reinforcement learning and derived improvements
In robotics, soft continuum robot arms are a promising prospect owing to their redundancy and passivity; however, no comprehensive study exists that examines their characteristics compared to rigid manipulators. In this study, we examined the advantages of a continuum robot arm as compared to a typi...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-09-01
|
| Series: | Frontiers in Robotics and AI |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frobt.2022.895388/full |
| _version_ | 1811337658789527552 |
|---|---|
| author | Ryota Morimoto Masahiro Ikeda Ryuma Niiyama Yasuo Kuniyoshi |
| author_facet | Ryota Morimoto Masahiro Ikeda Ryuma Niiyama Yasuo Kuniyoshi |
| author_sort | Ryota Morimoto |
| collection | DOAJ |
| description | In robotics, soft continuum robot arms are a promising prospect owing to their redundancy and passivity; however, no comprehensive study exists that examines their characteristics compared to rigid manipulators. In this study, we examined the advantages of a continuum robot arm as compared to a typical and rigid seven-degree-of-freedom (7-DoF) robot manipulator in terms of performing various tasks through reinforcement learning. We conducted simulations for tasks with different characteristics that require control over position and force. Common tasks in robot manipulators, such as reaching, crank rotation, object throwing, and peg-in-hole were considered. The initial conditions of the robot and environment were randomized, aiming for evaluations including robustness. The results indicate that the continuum robot arm excels in the crank-rotation task, which is characterized by uncertainty in environmental conditions and cumulative rewards. However, the rigid robot arm learned better motions for the peg-in-hole task than the other tasks, which requires fine motion control of the end-effector. In the throwing task, the continuum robot arm scored well owing to its good handling of anisotropy. Moreover, we developed a reinforcement-learning method based on the comprehensive experimental results. The proposed method successfully improved the motion learning of a continuum robot arm by adding a technique to regulate the initial state of the robot. To the best of our knowledge, ours is the first reinforcement-learning experiment with multiple tasks on a single continuum robot arm and is the first report of a comparison between a single continuum robot arm and rigid manipulator on a wide range of tasks. This simulation study can make a significant contribution to the design of continuum arms and specification of their applications, and development of control and reinforcement learning methods. |
| first_indexed | 2024-04-13T17:58:01Z |
| format | Article |
| id | doaj.art-8bb737001ced4d1fab19a07eb98a333f |
| institution | Directory Open Access Journal |
| issn | 2296-9144 |
| language | English |
| last_indexed | 2024-04-13T17:58:01Z |
| publishDate | 2022-09-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Robotics and AI |
| spelling | doaj.art-8bb737001ced4d1fab19a07eb98a333f2022-12-22T02:36:24ZengFrontiers Media S.A.Frontiers in Robotics and AI2296-91442022-09-01910.3389/frobt.2022.895388895388Characterization of continuum robot arms under reinforcement learning and derived improvementsRyota MorimotoMasahiro IkedaRyuma NiiyamaYasuo KuniyoshiIn robotics, soft continuum robot arms are a promising prospect owing to their redundancy and passivity; however, no comprehensive study exists that examines their characteristics compared to rigid manipulators. In this study, we examined the advantages of a continuum robot arm as compared to a typical and rigid seven-degree-of-freedom (7-DoF) robot manipulator in terms of performing various tasks through reinforcement learning. We conducted simulations for tasks with different characteristics that require control over position and force. Common tasks in robot manipulators, such as reaching, crank rotation, object throwing, and peg-in-hole were considered. The initial conditions of the robot and environment were randomized, aiming for evaluations including robustness. The results indicate that the continuum robot arm excels in the crank-rotation task, which is characterized by uncertainty in environmental conditions and cumulative rewards. However, the rigid robot arm learned better motions for the peg-in-hole task than the other tasks, which requires fine motion control of the end-effector. In the throwing task, the continuum robot arm scored well owing to its good handling of anisotropy. Moreover, we developed a reinforcement-learning method based on the comprehensive experimental results. The proposed method successfully improved the motion learning of a continuum robot arm by adding a technique to regulate the initial state of the robot. To the best of our knowledge, ours is the first reinforcement-learning experiment with multiple tasks on a single continuum robot arm and is the first report of a comparison between a single continuum robot arm and rigid manipulator on a wide range of tasks. This simulation study can make a significant contribution to the design of continuum arms and specification of their applications, and development of control and reinforcement learning methods.https://www.frontiersin.org/articles/10.3389/frobt.2022.895388/fullsoft roboticscontinuum robot armreinforcement learningreachingcrank rotationthrowing |
| spellingShingle | Ryota Morimoto Masahiro Ikeda Ryuma Niiyama Yasuo Kuniyoshi Characterization of continuum robot arms under reinforcement learning and derived improvements Frontiers in Robotics and AI soft robotics continuum robot arm reinforcement learning reaching crank rotation throwing |
| title | Characterization of continuum robot arms under reinforcement learning and derived improvements |
| title_full | Characterization of continuum robot arms under reinforcement learning and derived improvements |
| title_fullStr | Characterization of continuum robot arms under reinforcement learning and derived improvements |
| title_full_unstemmed | Characterization of continuum robot arms under reinforcement learning and derived improvements |
| title_short | Characterization of continuum robot arms under reinforcement learning and derived improvements |
| title_sort | characterization of continuum robot arms under reinforcement learning and derived improvements |
| topic | soft robotics continuum robot arm reinforcement learning reaching crank rotation throwing |
| url | https://www.frontiersin.org/articles/10.3389/frobt.2022.895388/full |
| work_keys_str_mv | AT ryotamorimoto characterizationofcontinuumrobotarmsunderreinforcementlearningandderivedimprovements AT masahiroikeda characterizationofcontinuumrobotarmsunderreinforcementlearningandderivedimprovements AT ryumaniiyama characterizationofcontinuumrobotarmsunderreinforcementlearningandderivedimprovements AT yasuokuniyoshi characterizationofcontinuumrobotarmsunderreinforcementlearningandderivedimprovements |