Target-Following Control of a Biomimetic Autonomous System Based on Predictive Reinforcement Learning
Biological fish often swim in a schooling manner, the mechanism of which comes from the fact that these schooling movements can improve the fishes’ hydrodynamic efficiency. Inspired by this phenomenon, a target-following control framework for a biomimetic autonomous system is proposed in this paper....
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-01-01
|
| Series: | Biomimetics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-7673/9/1/33 |
| _version_ | 1797344588225052672 |
|---|---|
| author | Yu Wang Jian Wang Song Kang Junzhi Yu |
| author_facet | Yu Wang Jian Wang Song Kang Junzhi Yu |
| author_sort | Yu Wang |
| collection | DOAJ |
| description | Biological fish often swim in a schooling manner, the mechanism of which comes from the fact that these schooling movements can improve the fishes’ hydrodynamic efficiency. Inspired by this phenomenon, a target-following control framework for a biomimetic autonomous system is proposed in this paper. Firstly, a following motion model is established based on the mechanism of fish schooling swimming, in which the follower robotic fish keeps a certain distance and orientation from the leader robotic fish. Second, by incorporating a predictive concept into reinforcement learning, a predictive deep deterministic policy gradient-following controller is provided with the normalized state space, action space, reward, and prediction design. It can avoid overshoot to a certain extent. A nonlinear model predictive controller is designed and can be selected for the follower robotic fish, together with the predictive reinforcement learning. Finally, extensive simulations are conducted, including the fix point and dynamic target following for single robotic fish, as well as cooperative following with the leader robotic fish. The obtained results indicate the effectiveness of the proposed methods, providing a valuable sight for the cooperative control of underwater robots to explore the ocean. |
| first_indexed | 2024-03-08T11:04:54Z |
| format | Article |
| id | doaj.art-f4c0682501bf4c15a40b8ead0b3b807b |
| institution | Directory Open Access Journal |
| issn | 2313-7673 |
| language | English |
| last_indexed | 2024-03-08T11:04:54Z |
| publishDate | 2024-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Biomimetics |
| spelling | doaj.art-f4c0682501bf4c15a40b8ead0b3b807b2024-01-26T15:15:50ZengMDPI AGBiomimetics2313-76732024-01-01913310.3390/biomimetics9010033Target-Following Control of a Biomimetic Autonomous System Based on Predictive Reinforcement LearningYu Wang0Jian Wang1Song Kang2Junzhi Yu3Department of Automation, Tsinghua University, Beijing 100084, ChinaThe Laboratory of Cognitive and Decision Intelligence for Complex System, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, ChinaThe Laboratory of Cognitive and Decision Intelligence for Complex System, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, ChinaThe Laboratory of Cognitive and Decision Intelligence for Complex System, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, ChinaBiological fish often swim in a schooling manner, the mechanism of which comes from the fact that these schooling movements can improve the fishes’ hydrodynamic efficiency. Inspired by this phenomenon, a target-following control framework for a biomimetic autonomous system is proposed in this paper. Firstly, a following motion model is established based on the mechanism of fish schooling swimming, in which the follower robotic fish keeps a certain distance and orientation from the leader robotic fish. Second, by incorporating a predictive concept into reinforcement learning, a predictive deep deterministic policy gradient-following controller is provided with the normalized state space, action space, reward, and prediction design. It can avoid overshoot to a certain extent. A nonlinear model predictive controller is designed and can be selected for the follower robotic fish, together with the predictive reinforcement learning. Finally, extensive simulations are conducted, including the fix point and dynamic target following for single robotic fish, as well as cooperative following with the leader robotic fish. The obtained results indicate the effectiveness of the proposed methods, providing a valuable sight for the cooperative control of underwater robots to explore the ocean.https://www.mdpi.com/2313-7673/9/1/33biomimetic motionbiomimetic autonomous systemtarget followingdeep reinforcement learningpredictive control |
| spellingShingle | Yu Wang Jian Wang Song Kang Junzhi Yu Target-Following Control of a Biomimetic Autonomous System Based on Predictive Reinforcement Learning Biomimetics biomimetic motion biomimetic autonomous system target following deep reinforcement learning predictive control |
| title | Target-Following Control of a Biomimetic Autonomous System Based on Predictive Reinforcement Learning |
| title_full | Target-Following Control of a Biomimetic Autonomous System Based on Predictive Reinforcement Learning |
| title_fullStr | Target-Following Control of a Biomimetic Autonomous System Based on Predictive Reinforcement Learning |
| title_full_unstemmed | Target-Following Control of a Biomimetic Autonomous System Based on Predictive Reinforcement Learning |
| title_short | Target-Following Control of a Biomimetic Autonomous System Based on Predictive Reinforcement Learning |
| title_sort | target following control of a biomimetic autonomous system based on predictive reinforcement learning |
| topic | biomimetic motion biomimetic autonomous system target following deep reinforcement learning predictive control |
| url | https://www.mdpi.com/2313-7673/9/1/33 |
| work_keys_str_mv | AT yuwang targetfollowingcontrolofabiomimeticautonomoussystembasedonpredictivereinforcementlearning AT jianwang targetfollowingcontrolofabiomimeticautonomoussystembasedonpredictivereinforcementlearning AT songkang targetfollowingcontrolofabiomimeticautonomoussystembasedonpredictivereinforcementlearning AT junzhiyu targetfollowingcontrolofabiomimeticautonomoussystembasedonpredictivereinforcementlearning |