Robotic Arm Trajectory Planning Method Using Deep Deterministic Policy Gradient With Hierarchical Memory Structure

Traditional robotic arm path planning methods are mainly carried out in the tool center point operation space, and frequently solve inverse kinematics problems, thus consuming a large number of computational resources. In contrast, the use of positive kinematics for planning in joint space not only...

Full description

Bibliographic Details
Main Authors: Di Zhao, Zhenyu Ding, Wenjie Li, Sen Zhao, Yuhong Du
Format: Article
Language:English
Published: IEEE 2023-01-01
Series:IEEE Access
Subjects:
Online Access:https://ieeexplore.ieee.org/document/10348581/
_version_ 1797376295080820736
author Di Zhao
Zhenyu Ding
Wenjie Li
Sen Zhao
Yuhong Du
author_facet Di Zhao
Zhenyu Ding
Wenjie Li
Sen Zhao
Yuhong Du
author_sort Di Zhao
collection DOAJ
description Traditional robotic arm path planning methods are mainly carried out in the tool center point operation space, and frequently solve inverse kinematics problems, thus consuming a large number of computational resources. In contrast, the use of positive kinematics for planning in joint space not only enhances efficiency, but also provides analytic solutions with higher accuracy. In order to better cover the environmental state space, this paper adopts the full-preserving experience preservation approach. In order to realize fast and efficient sampling of high reward value experience, this study constructs an innovative hierarchical memory structure and eliminates the overfitting phenomenon that may be caused by biased sampling through the Bias-Free strategy. Experimentally validated in the continuous path planning task of a textile robot arm, the proposed hierarchical memory deep deterministic gradient strategy method (HM-DDPG) demonstrates excellent performance and practicality in the textile robot arm path planning problem. This method offers an efficient and robust solution for handling complex tasks with temporal dependencies, paving the way for future innovations and applications in industrial fields such as textiles.
first_indexed 2024-03-08T19:36:29Z
format Article
id doaj.art-8e956913de214c90b48b276529b14598
institution Directory Open Access Journal
issn 2169-3536
language English
last_indexed 2024-03-08T19:36:29Z
publishDate 2023-01-01
publisher IEEE
record_format Article
series IEEE Access
spelling doaj.art-8e956913de214c90b48b276529b145982023-12-26T00:10:16ZengIEEEIEEE Access2169-35362023-01-011114080114081410.1109/ACCESS.2023.334068410348581Robotic Arm Trajectory Planning Method Using Deep Deterministic Policy Gradient With Hierarchical Memory StructureDi Zhao0https://orcid.org/0009-0001-5969-2836Zhenyu Ding1Wenjie Li2https://orcid.org/0009-0008-6119-487XSen Zhao3Yuhong Du4https://orcid.org/0009-0007-6965-3105School of Mechanical Engineering, Tiangong University, Tianjin, ChinaSchool of Electronics and Information Engineering, Tiangong University, Tianjin, ChinaSchool of Mechanical Engineering, Tiangong University, Tianjin, ChinaSchool of Mechanical Engineering, Tiangong University, Tianjin, ChinaInnovation College, Tiangong University, Tianjin, ChinaTraditional robotic arm path planning methods are mainly carried out in the tool center point operation space, and frequently solve inverse kinematics problems, thus consuming a large number of computational resources. In contrast, the use of positive kinematics for planning in joint space not only enhances efficiency, but also provides analytic solutions with higher accuracy. In order to better cover the environmental state space, this paper adopts the full-preserving experience preservation approach. In order to realize fast and efficient sampling of high reward value experience, this study constructs an innovative hierarchical memory structure and eliminates the overfitting phenomenon that may be caused by biased sampling through the Bias-Free strategy. Experimentally validated in the continuous path planning task of a textile robot arm, the proposed hierarchical memory deep deterministic gradient strategy method (HM-DDPG) demonstrates excellent performance and practicality in the textile robot arm path planning problem. This method offers an efficient and robust solution for handling complex tasks with temporal dependencies, paving the way for future innovations and applications in industrial fields such as textiles.https://ieeexplore.ieee.org/document/10348581/Continuous control taskdeep deterministic policy gradientpath planningpositive kinematicshierarchical memory
spellingShingle Di Zhao
Zhenyu Ding
Wenjie Li
Sen Zhao
Yuhong Du
Robotic Arm Trajectory Planning Method Using Deep Deterministic Policy Gradient With Hierarchical Memory Structure
IEEE Access
Continuous control task
deep deterministic policy gradient
path planning
positive kinematics
hierarchical memory
title Robotic Arm Trajectory Planning Method Using Deep Deterministic Policy Gradient With Hierarchical Memory Structure
title_full Robotic Arm Trajectory Planning Method Using Deep Deterministic Policy Gradient With Hierarchical Memory Structure
title_fullStr Robotic Arm Trajectory Planning Method Using Deep Deterministic Policy Gradient With Hierarchical Memory Structure
title_full_unstemmed Robotic Arm Trajectory Planning Method Using Deep Deterministic Policy Gradient With Hierarchical Memory Structure
title_short Robotic Arm Trajectory Planning Method Using Deep Deterministic Policy Gradient With Hierarchical Memory Structure
title_sort robotic arm trajectory planning method using deep deterministic policy gradient with hierarchical memory structure
topic Continuous control task
deep deterministic policy gradient
path planning
positive kinematics
hierarchical memory
url https://ieeexplore.ieee.org/document/10348581/
work_keys_str_mv AT dizhao roboticarmtrajectoryplanningmethodusingdeepdeterministicpolicygradientwithhierarchicalmemorystructure
AT zhenyuding roboticarmtrajectoryplanningmethodusingdeepdeterministicpolicygradientwithhierarchicalmemorystructure
AT wenjieli roboticarmtrajectoryplanningmethodusingdeepdeterministicpolicygradientwithhierarchicalmemorystructure
AT senzhao roboticarmtrajectoryplanningmethodusingdeepdeterministicpolicygradientwithhierarchicalmemorystructure
AT yuhongdu roboticarmtrajectoryplanningmethodusingdeepdeterministicpolicygradientwithhierarchicalmemorystructure