Stable training via elastic adaptive deep reinforcement learning for autonomous navigation of intelligent vehicles
Abstract The uncertain stability of deep reinforcement learning training on complex tasks impedes its development and deployment, especially in intelligent vehicles, such as intelligent surface vessels and self-driving cars. Complex and varied environmental states puzzle training of decision-making...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-02-01
|
| Series: | Communications Engineering |
| Online Access: | https://doi.org/10.1038/s44172-024-00182-8 |
| _version_ | 1797247276553338880 |
|---|---|
| author | Yujiao Zhao Yong Ma Guibing Zhu Songlin Hu Xinping Yan |
| author_facet | Yujiao Zhao Yong Ma Guibing Zhu Songlin Hu Xinping Yan |
| author_sort | Yujiao Zhao |
| collection | DOAJ |
| description | Abstract The uncertain stability of deep reinforcement learning training on complex tasks impedes its development and deployment, especially in intelligent vehicles, such as intelligent surface vessels and self-driving cars. Complex and varied environmental states puzzle training of decision-making networks. Here we propose an elastic adaptive deep reinforcement learning algorithm to address these challenges and achieve autonomous navigation in intelligent vehicles. Our method trains the decision-making network over the function and optimization learning stages, in which the state space and action space of autonomous navigation tasks are pruned by choosing classic states and actions to reduce data similarity, facilitating more stable training. We introduce a task-adaptive observed behaviour classification technique in the function learning stage to divide state and action spaces into subspaces and identify classic states and actions. In which the classic states and actions are accumulated as the training dataset that enhances its training efficiency. In the subsequent optimization learning stage, the decision-making network is refined through meticulous exploration and accumulation of datasets. The proposed elastic adaptive deep reinforcement learning enables the decision-making network to effectively learn from complex state and action spaces, leading to more efficient training compared to traditional deep reinforcement learning approaches. Simulation results demonstrate the remarkable effectiveness of our method in training decision-making networks for intelligent vehicles. The findings validate that our method provides reliable and efficient training for decision-making networks in intelligent vehicles. Moreover, our method exhibits stability in training other tasks characterized by continuous state and action spaces. |
| first_indexed | 2024-03-07T14:58:34Z |
| format | Article |
| id | doaj.art-8ddf5510da7d49039b753259988ee509 |
| institution | Directory Open Access Journal |
| issn | 2731-3395 |
| language | English |
| last_indexed | 2024-04-24T19:56:07Z |
| publishDate | 2024-02-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Communications Engineering |
| spelling | doaj.art-8ddf5510da7d49039b753259988ee5092024-03-24T12:22:17ZengNature PortfolioCommunications Engineering2731-33952024-02-01311810.1038/s44172-024-00182-8Stable training via elastic adaptive deep reinforcement learning for autonomous navigation of intelligent vehiclesYujiao Zhao0Yong Ma1Guibing Zhu2Songlin Hu3Xinping Yan4State Key Laboratory of Maritime Technology and Safety, Wuhan University of TechnologyState Key Laboratory of Maritime Technology and Safety, Wuhan University of TechnologyMarine College, Zhejiang Ocean UniversityInstitute of Advanced Technology, Nanjing University of Posts and TelecommunicationsState Key Laboratory of Maritime Technology and Safety, Wuhan University of TechnologyAbstract The uncertain stability of deep reinforcement learning training on complex tasks impedes its development and deployment, especially in intelligent vehicles, such as intelligent surface vessels and self-driving cars. Complex and varied environmental states puzzle training of decision-making networks. Here we propose an elastic adaptive deep reinforcement learning algorithm to address these challenges and achieve autonomous navigation in intelligent vehicles. Our method trains the decision-making network over the function and optimization learning stages, in which the state space and action space of autonomous navigation tasks are pruned by choosing classic states and actions to reduce data similarity, facilitating more stable training. We introduce a task-adaptive observed behaviour classification technique in the function learning stage to divide state and action spaces into subspaces and identify classic states and actions. In which the classic states and actions are accumulated as the training dataset that enhances its training efficiency. In the subsequent optimization learning stage, the decision-making network is refined through meticulous exploration and accumulation of datasets. The proposed elastic adaptive deep reinforcement learning enables the decision-making network to effectively learn from complex state and action spaces, leading to more efficient training compared to traditional deep reinforcement learning approaches. Simulation results demonstrate the remarkable effectiveness of our method in training decision-making networks for intelligent vehicles. The findings validate that our method provides reliable and efficient training for decision-making networks in intelligent vehicles. Moreover, our method exhibits stability in training other tasks characterized by continuous state and action spaces.https://doi.org/10.1038/s44172-024-00182-8 |
| spellingShingle | Yujiao Zhao Yong Ma Guibing Zhu Songlin Hu Xinping Yan Stable training via elastic adaptive deep reinforcement learning for autonomous navigation of intelligent vehicles Communications Engineering |
| title | Stable training via elastic adaptive deep reinforcement learning for autonomous navigation of intelligent vehicles |
| title_full | Stable training via elastic adaptive deep reinforcement learning for autonomous navigation of intelligent vehicles |
| title_fullStr | Stable training via elastic adaptive deep reinforcement learning for autonomous navigation of intelligent vehicles |
| title_full_unstemmed | Stable training via elastic adaptive deep reinforcement learning for autonomous navigation of intelligent vehicles |
| title_short | Stable training via elastic adaptive deep reinforcement learning for autonomous navigation of intelligent vehicles |
| title_sort | stable training via elastic adaptive deep reinforcement learning for autonomous navigation of intelligent vehicles |
| url | https://doi.org/10.1038/s44172-024-00182-8 |
| work_keys_str_mv | AT yujiaozhao stabletrainingviaelasticadaptivedeepreinforcementlearningforautonomousnavigationofintelligentvehicles AT yongma stabletrainingviaelasticadaptivedeepreinforcementlearningforautonomousnavigationofintelligentvehicles AT guibingzhu stabletrainingviaelasticadaptivedeepreinforcementlearningforautonomousnavigationofintelligentvehicles AT songlinhu stabletrainingviaelasticadaptivedeepreinforcementlearningforautonomousnavigationofintelligentvehicles AT xinpingyan stabletrainingviaelasticadaptivedeepreinforcementlearningforautonomousnavigationofintelligentvehicles |