A Deep Reinforcement-Learning Approach for Inverse Kinematics Solution of a High Degree of Freedom Robotic Manipulator
The foundation and emphasis of robotic manipulator control is Inverse Kinematics (IK). Due to the complexity of derivation, difficulty of computation, and redundancy, traditional IK solutions pose numerous challenges to the operation of a variety of robotic manipulators. This paper develops a Deep R...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-04-01
|
| Series: | Robotics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2218-6581/11/2/44 |
| _version_ | 1797434252164333568 |
|---|---|
| author | Aryslan Malik Yevgeniy Lischuk Troy Henderson Richard Prazenica |
| author_facet | Aryslan Malik Yevgeniy Lischuk Troy Henderson Richard Prazenica |
| author_sort | Aryslan Malik |
| collection | DOAJ |
| description | The foundation and emphasis of robotic manipulator control is Inverse Kinematics (IK). Due to the complexity of derivation, difficulty of computation, and redundancy, traditional IK solutions pose numerous challenges to the operation of a variety of robotic manipulators. This paper develops a Deep Reinforcement Learning (RL) approach for solving the IK problem of a 7-Degree of Freedom (DOF) robotic manipulator using Product of Exponentials (PoE) as a Forward Kinematics (FK) computation tool and the Deep Q-Network (DQN) as an IK solver. The selected approach is architecturally simpler, making it faster and easier to implement, as well as more stable, because it is less sensitive to hyperparameters than continuous action spaces algorithms. The algorithm is designed to produce joint-space trajectories from a given end-effector trajectory. Different network architectures were explored and the output of the DQN was implemented experimentally on a Sawyer robotic arm. The DQN was able to find different trajectories corresponding to a specified Cartesian path of the end-effector. The network agent was able to learn random Bézier and straight-line end-effector trajectories in a reasonable time frame with good accuracy, demonstrating that even though DQN is mainly used in discrete solution spaces, it could be applied to generate joint space trajectories. |
| first_indexed | 2024-03-09T10:30:31Z |
| format | Article |
| id | doaj.art-d8213e1d3313498caac5b4457ec63798 |
| institution | Directory Open Access Journal |
| issn | 2218-6581 |
| language | English |
| last_indexed | 2024-03-09T10:30:31Z |
| publishDate | 2022-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Robotics |
| spelling | doaj.art-d8213e1d3313498caac5b4457ec637982023-12-01T21:22:17ZengMDPI AGRobotics2218-65812022-04-011124410.3390/robotics11020044A Deep Reinforcement-Learning Approach for Inverse Kinematics Solution of a High Degree of Freedom Robotic ManipulatorAryslan Malik0Yevgeniy Lischuk1Troy Henderson2Richard Prazenica3Aerospace Engineering Department, Embry—Riddle Aeronautical University, Daytona Beach, FL 32114, USASoftware—Device OS, Amazon, Austin, TX 78758, USAAerospace Engineering Department, Embry—Riddle Aeronautical University, Daytona Beach, FL 32114, USAAerospace Engineering Department, Embry—Riddle Aeronautical University, Daytona Beach, FL 32114, USAThe foundation and emphasis of robotic manipulator control is Inverse Kinematics (IK). Due to the complexity of derivation, difficulty of computation, and redundancy, traditional IK solutions pose numerous challenges to the operation of a variety of robotic manipulators. This paper develops a Deep Reinforcement Learning (RL) approach for solving the IK problem of a 7-Degree of Freedom (DOF) robotic manipulator using Product of Exponentials (PoE) as a Forward Kinematics (FK) computation tool and the Deep Q-Network (DQN) as an IK solver. The selected approach is architecturally simpler, making it faster and easier to implement, as well as more stable, because it is less sensitive to hyperparameters than continuous action spaces algorithms. The algorithm is designed to produce joint-space trajectories from a given end-effector trajectory. Different network architectures were explored and the output of the DQN was implemented experimentally on a Sawyer robotic arm. The DQN was able to find different trajectories corresponding to a specified Cartesian path of the end-effector. The network agent was able to learn random Bézier and straight-line end-effector trajectories in a reasonable time frame with good accuracy, demonstrating that even though DQN is mainly used in discrete solution spaces, it could be applied to generate joint space trajectories.https://www.mdpi.com/2218-6581/11/2/44roboticsrobotic manipulatorInverse Kinematicsneural networksReinforcement LearningDeep Learning |
| spellingShingle | Aryslan Malik Yevgeniy Lischuk Troy Henderson Richard Prazenica A Deep Reinforcement-Learning Approach for Inverse Kinematics Solution of a High Degree of Freedom Robotic Manipulator Robotics robotics robotic manipulator Inverse Kinematics neural networks Reinforcement Learning Deep Learning |
| title | A Deep Reinforcement-Learning Approach for Inverse Kinematics Solution of a High Degree of Freedom Robotic Manipulator |
| title_full | A Deep Reinforcement-Learning Approach for Inverse Kinematics Solution of a High Degree of Freedom Robotic Manipulator |
| title_fullStr | A Deep Reinforcement-Learning Approach for Inverse Kinematics Solution of a High Degree of Freedom Robotic Manipulator |
| title_full_unstemmed | A Deep Reinforcement-Learning Approach for Inverse Kinematics Solution of a High Degree of Freedom Robotic Manipulator |
| title_short | A Deep Reinforcement-Learning Approach for Inverse Kinematics Solution of a High Degree of Freedom Robotic Manipulator |
| title_sort | deep reinforcement learning approach for inverse kinematics solution of a high degree of freedom robotic manipulator |
| topic | robotics robotic manipulator Inverse Kinematics neural networks Reinforcement Learning Deep Learning |
| url | https://www.mdpi.com/2218-6581/11/2/44 |
| work_keys_str_mv | AT aryslanmalik adeepreinforcementlearningapproachforinversekinematicssolutionofahighdegreeoffreedomroboticmanipulator AT yevgeniylischuk adeepreinforcementlearningapproachforinversekinematicssolutionofahighdegreeoffreedomroboticmanipulator AT troyhenderson adeepreinforcementlearningapproachforinversekinematicssolutionofahighdegreeoffreedomroboticmanipulator AT richardprazenica adeepreinforcementlearningapproachforinversekinematicssolutionofahighdegreeoffreedomroboticmanipulator AT aryslanmalik deepreinforcementlearningapproachforinversekinematicssolutionofahighdegreeoffreedomroboticmanipulator AT yevgeniylischuk deepreinforcementlearningapproachforinversekinematicssolutionofahighdegreeoffreedomroboticmanipulator AT troyhenderson deepreinforcementlearningapproachforinversekinematicssolutionofahighdegreeoffreedomroboticmanipulator AT richardprazenica deepreinforcementlearningapproachforinversekinematicssolutionofahighdegreeoffreedomroboticmanipulator |