Buoy and Winch Collaborative Control System Based on Deep Reinforcement Learning
The improved control performance of the buoy and winch collaborative control system can enhance the stability of the connection between underwater robots and ground industrial control equipment. To overcome the challenge of mathematical modeling of this control system, this research introduces reinf...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-02-01
|
| Series: | Journal of Marine Science and Engineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-1312/13/2/326 |
| _version_ | 1826582229050982400 |
|---|---|
| author | Yang Gu Jianjun Ni Zaiming Geng Bing Zhao Haowen Yang |
| author_facet | Yang Gu Jianjun Ni Zaiming Geng Bing Zhao Haowen Yang |
| author_sort | Yang Gu |
| collection | DOAJ |
| description | The improved control performance of the buoy and winch collaborative control system can enhance the stability of the connection between underwater robots and ground industrial control equipment. To overcome the challenge of mathematical modeling of this control system, this research introduces reinforcement learning and transformer models in the design process. The main contributions include the development of two simulation environments for training DRL agents, designing a reward function to guide the exploration process, proposing a buoy control algorithm based on the discrete soft actor-critic (SAC) algorithm, and proposing a winch cable length prediction algorithm based on a lightweight transformer model. The experiment results demonstrated significant improvements in rewards diagrams, buoy control trajectories, and winch model performance, showcasing the effectiveness of our proposed system. The average error of the buoy tracking trajectories induced by different policies trained in the two environments is less than 0.05, and the evaluation error of the behavior cloning lightweight transformer model is less than 0.03. |
| first_indexed | 2025-03-14T15:02:38Z |
| format | Article |
| id | doaj.art-2047dfb43000449cb26d7143b1320f4b |
| institution | Directory Open Access Journal |
| issn | 2077-1312 |
| language | English |
| last_indexed | 2025-03-14T15:02:38Z |
| publishDate | 2025-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Marine Science and Engineering |
| spelling | doaj.art-2047dfb43000449cb26d7143b1320f4b2025-02-25T13:33:45ZengMDPI AGJournal of Marine Science and Engineering2077-13122025-02-0113232610.3390/jmse13020326Buoy and Winch Collaborative Control System Based on Deep Reinforcement LearningYang Gu0Jianjun Ni1Zaiming Geng2Bing Zhao3Haowen Yang4College of Artificial Intelligence and Automation, Hohai University, Changzhou 213200, ChinaCollege of Artificial Intelligence and Automation, Hohai University, Changzhou 213200, ChinaChina Yangtze Power Co., Ltd., Yichang 443002, ChinaShenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, ChinaSchool of Science and Engineering, The Chinese University of Hong Kong, Shenzhen 518172, ChinaThe improved control performance of the buoy and winch collaborative control system can enhance the stability of the connection between underwater robots and ground industrial control equipment. To overcome the challenge of mathematical modeling of this control system, this research introduces reinforcement learning and transformer models in the design process. The main contributions include the development of two simulation environments for training DRL agents, designing a reward function to guide the exploration process, proposing a buoy control algorithm based on the discrete soft actor-critic (SAC) algorithm, and proposing a winch cable length prediction algorithm based on a lightweight transformer model. The experiment results demonstrated significant improvements in rewards diagrams, buoy control trajectories, and winch model performance, showcasing the effectiveness of our proposed system. The average error of the buoy tracking trajectories induced by different policies trained in the two environments is less than 0.05, and the evaluation error of the behavior cloning lightweight transformer model is less than 0.03.https://www.mdpi.com/2077-1312/13/2/326buoy and winch control systemsoft actor-critictransformerbehavior cloning |
| spellingShingle | Yang Gu Jianjun Ni Zaiming Geng Bing Zhao Haowen Yang Buoy and Winch Collaborative Control System Based on Deep Reinforcement Learning Journal of Marine Science and Engineering buoy and winch control system soft actor-critic transformer behavior cloning |
| title | Buoy and Winch Collaborative Control System Based on Deep Reinforcement Learning |
| title_full | Buoy and Winch Collaborative Control System Based on Deep Reinforcement Learning |
| title_fullStr | Buoy and Winch Collaborative Control System Based on Deep Reinforcement Learning |
| title_full_unstemmed | Buoy and Winch Collaborative Control System Based on Deep Reinforcement Learning |
| title_short | Buoy and Winch Collaborative Control System Based on Deep Reinforcement Learning |
| title_sort | buoy and winch collaborative control system based on deep reinforcement learning |
| topic | buoy and winch control system soft actor-critic transformer behavior cloning |
| url | https://www.mdpi.com/2077-1312/13/2/326 |
| work_keys_str_mv | AT yanggu buoyandwinchcollaborativecontrolsystembasedondeepreinforcementlearning AT jianjunni buoyandwinchcollaborativecontrolsystembasedondeepreinforcementlearning AT zaiminggeng buoyandwinchcollaborativecontrolsystembasedondeepreinforcementlearning AT bingzhao buoyandwinchcollaborativecontrolsystembasedondeepreinforcementlearning AT haowenyang buoyandwinchcollaborativecontrolsystembasedondeepreinforcementlearning |