A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots
When a mobile robot inspects tasks with complex requirements indoors, the traditional backstepping method cannot guarantee the accuracy of the trajectory, leading to problems such as the instrument not being inside the image and focus failure when the robot grabs the image with high zoom. In order t...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Actuators |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-0825/12/8/326 |
| _version_ | 1797585868879298560 |
|---|---|
| author | Naifeng He Zhong Yang Xiaoliang Fan Jiying Wu Yaoyu Sui Qiuyan Zhang |
| author_facet | Naifeng He Zhong Yang Xiaoliang Fan Jiying Wu Yaoyu Sui Qiuyan Zhang |
| author_sort | Naifeng He |
| collection | DOAJ |
| description | When a mobile robot inspects tasks with complex requirements indoors, the traditional backstepping method cannot guarantee the accuracy of the trajectory, leading to problems such as the instrument not being inside the image and focus failure when the robot grabs the image with high zoom. In order to solve this problem, this paper proposes an adaptive backstepping method based on double Q-learning for tracking and controlling the trajectory of mobile robots. We design the incremental model-free algorithm of Double-Q learning, which can quickly learn to rectify the trajectory tracking controller gain online. For the controller gain rectification problem in non-uniform state space exploration, we propose an incremental active learning exploration algorithm that incorporates memory playback as well as experience playback mechanisms to achieve online fast learning and controller gain rectification for agents. To verify the feasibility of the algorithm, we perform algorithm verification on different types of trajectories in Gazebo and physical platforms. The results show that the adaptive trajectory tracking control algorithm can be used to rectify the mobile robot trajectory tracking controller’s gain. Compared with the Backstepping-Fractional-Older PID controller and Fuzzy-Backstepping controller, Double Q-backstepping has better robustness, generalization, real-time, and stronger anti-disturbance capability. |
| first_indexed | 2024-03-11T00:13:15Z |
| format | Article |
| id | doaj.art-2f0141187cf24d8ebc0501dbe90e3cb5 |
| institution | Directory Open Access Journal |
| issn | 2076-0825 |
| language | English |
| last_indexed | 2024-03-11T00:13:15Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Actuators |
| spelling | doaj.art-2f0141187cf24d8ebc0501dbe90e3cb52023-11-18T23:49:01ZengMDPI AGActuators2076-08252023-08-0112832610.3390/act12080326A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile RobotsNaifeng He0Zhong Yang1Xiaoliang Fan2Jiying Wu3Yaoyu Sui4Qiuyan Zhang5College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaState Key Laboratory of Robotics, Shenyang Institute of Automation Chinese Academy of Sciences, Shenyang 110017, ChinaCollege of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaElectric Power Research Institute of Guizhou Power Grid Co., Ltd., Guiyang 550002, ChinaWhen a mobile robot inspects tasks with complex requirements indoors, the traditional backstepping method cannot guarantee the accuracy of the trajectory, leading to problems such as the instrument not being inside the image and focus failure when the robot grabs the image with high zoom. In order to solve this problem, this paper proposes an adaptive backstepping method based on double Q-learning for tracking and controlling the trajectory of mobile robots. We design the incremental model-free algorithm of Double-Q learning, which can quickly learn to rectify the trajectory tracking controller gain online. For the controller gain rectification problem in non-uniform state space exploration, we propose an incremental active learning exploration algorithm that incorporates memory playback as well as experience playback mechanisms to achieve online fast learning and controller gain rectification for agents. To verify the feasibility of the algorithm, we perform algorithm verification on different types of trajectories in Gazebo and physical platforms. The results show that the adaptive trajectory tracking control algorithm can be used to rectify the mobile robot trajectory tracking controller’s gain. Compared with the Backstepping-Fractional-Older PID controller and Fuzzy-Backstepping controller, Double Q-backstepping has better robustness, generalization, real-time, and stronger anti-disturbance capability.https://www.mdpi.com/2076-0825/12/8/326reinforcement learningdouble Q-backstepping controlmobile robottrajectory tracking control |
| spellingShingle | Naifeng He Zhong Yang Xiaoliang Fan Jiying Wu Yaoyu Sui Qiuyan Zhang A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots Actuators reinforcement learning double Q-backstepping control mobile robot trajectory tracking control |
| title | A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots |
| title_full | A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots |
| title_fullStr | A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots |
| title_full_unstemmed | A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots |
| title_short | A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots |
| title_sort | self adaptive double q backstepping trajectory tracking control approach based on reinforcement learning for mobile robots |
| topic | reinforcement learning double Q-backstepping control mobile robot trajectory tracking control |
| url | https://www.mdpi.com/2076-0825/12/8/326 |
| work_keys_str_mv | AT naifenghe aselfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT zhongyang aselfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT xiaoliangfan aselfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT jiyingwu aselfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT yaoyusui aselfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT qiuyanzhang aselfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT naifenghe selfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT zhongyang selfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT xiaoliangfan selfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT jiyingwu selfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT yaoyusui selfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots AT qiuyanzhang selfadaptivedoubleqbacksteppingtrajectorytrackingcontrolapproachbasedonreinforcementlearningformobilerobots |