Graph Reinforcement Learning-Based Decision-Making Technology for Connected and Autonomous Vehicles: Framework, Review, and Future Trends
The proper functioning of connected and autonomous vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. T...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/19/8229 |
| _version_ | 1797575166037852160 |
|---|---|
| author | Qi Liu Xueyuan Li Yujie Tang Xin Gao Fan Yang Zirui Li |
| author_facet | Qi Liu Xueyuan Li Yujie Tang Xin Gao Fan Yang Zirui Li |
| author_sort | Qi Liu |
| collection | DOAJ |
| description | The proper functioning of connected and autonomous vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. Thus, collaborative decision-making technology for CAVs is essential to generate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomy traffic. In recent years, deep reinforcement learning (DRL) methods have become an efficient way in solving decision-making problems. However, with the development of computing technology, graph reinforcement learning (GRL) methods have gradually demonstrated the large potential to further improve the decision-making performance of CAVs, especially in the area of accurately representing the mutual effects of vehicles and modeling dynamic traffic environments. To facilitate the development of GRL-based methods for autonomous driving, this paper proposes a review of GRL-based methods for the decision-making technologies of CAVs. Firstly, a generic GRL framework is proposed in the beginning to gain an overall understanding of the decision-making technology. Then, the GRL-based decision-making technologies are reviewed from the perspective of the construction methods of mixed autonomy traffic, methods for graph representation of the driving environment, and related works about graph neural networks (GNN) and DRL in the field of decision-making for autonomous driving. Moreover, validation methods are summarized to provide an efficient way to verify the performance of decision-making methods. Finally, challenges and future research directions of GRL-based decision-making methods are summarized. |
| first_indexed | 2024-03-10T21:34:47Z |
| format | Article |
| id | doaj.art-8a86c4cb95154f278331bf4509f74172 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T21:34:47Z |
| publishDate | 2023-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-8a86c4cb95154f278331bf4509f741722023-11-19T15:04:38ZengMDPI AGSensors1424-82202023-10-012319822910.3390/s23198229Graph Reinforcement Learning-Based Decision-Making Technology for Connected and Autonomous Vehicles: Framework, Review, and Future TrendsQi Liu0Xueyuan Li1Yujie Tang2Xin Gao3Fan Yang4Zirui Li5School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100811, ChinaSchool of Mechanical Engineering, Beijing Institute of Technology, Beijing 100811, ChinaFaculty of Computer Science, Dalhousie University, Halifax, NS B3H 4R2, CanadaSchool of Mechanical Engineering, Beijing Institute of Technology, Beijing 100811, ChinaSchool of Mechanical Engineering, Beijing Institute of Technology, Beijing 100811, ChinaSchool of Mechanical Engineering, Beijing Institute of Technology, Beijing 100811, ChinaThe proper functioning of connected and autonomous vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. Thus, collaborative decision-making technology for CAVs is essential to generate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomy traffic. In recent years, deep reinforcement learning (DRL) methods have become an efficient way in solving decision-making problems. However, with the development of computing technology, graph reinforcement learning (GRL) methods have gradually demonstrated the large potential to further improve the decision-making performance of CAVs, especially in the area of accurately representing the mutual effects of vehicles and modeling dynamic traffic environments. To facilitate the development of GRL-based methods for autonomous driving, this paper proposes a review of GRL-based methods for the decision-making technologies of CAVs. Firstly, a generic GRL framework is proposed in the beginning to gain an overall understanding of the decision-making technology. Then, the GRL-based decision-making technologies are reviewed from the perspective of the construction methods of mixed autonomy traffic, methods for graph representation of the driving environment, and related works about graph neural networks (GNN) and DRL in the field of decision-making for autonomous driving. Moreover, validation methods are summarized to provide an efficient way to verify the performance of decision-making methods. Finally, challenges and future research directions of GRL-based decision-making methods are summarized.https://www.mdpi.com/1424-8220/23/19/8229connected and autonomous vehiclegraph reinforcement learningdecision-makingmixed autonomy traffic |
| spellingShingle | Qi Liu Xueyuan Li Yujie Tang Xin Gao Fan Yang Zirui Li Graph Reinforcement Learning-Based Decision-Making Technology for Connected and Autonomous Vehicles: Framework, Review, and Future Trends Sensors connected and autonomous vehicle graph reinforcement learning decision-making mixed autonomy traffic |
| title | Graph Reinforcement Learning-Based Decision-Making Technology for Connected and Autonomous Vehicles: Framework, Review, and Future Trends |
| title_full | Graph Reinforcement Learning-Based Decision-Making Technology for Connected and Autonomous Vehicles: Framework, Review, and Future Trends |
| title_fullStr | Graph Reinforcement Learning-Based Decision-Making Technology for Connected and Autonomous Vehicles: Framework, Review, and Future Trends |
| title_full_unstemmed | Graph Reinforcement Learning-Based Decision-Making Technology for Connected and Autonomous Vehicles: Framework, Review, and Future Trends |
| title_short | Graph Reinforcement Learning-Based Decision-Making Technology for Connected and Autonomous Vehicles: Framework, Review, and Future Trends |
| title_sort | graph reinforcement learning based decision making technology for connected and autonomous vehicles framework review and future trends |
| topic | connected and autonomous vehicle graph reinforcement learning decision-making mixed autonomy traffic |
| url | https://www.mdpi.com/1424-8220/23/19/8229 |
| work_keys_str_mv | AT qiliu graphreinforcementlearningbaseddecisionmakingtechnologyforconnectedandautonomousvehiclesframeworkreviewandfuturetrends AT xueyuanli graphreinforcementlearningbaseddecisionmakingtechnologyforconnectedandautonomousvehiclesframeworkreviewandfuturetrends AT yujietang graphreinforcementlearningbaseddecisionmakingtechnologyforconnectedandautonomousvehiclesframeworkreviewandfuturetrends AT xingao graphreinforcementlearningbaseddecisionmakingtechnologyforconnectedandautonomousvehiclesframeworkreviewandfuturetrends AT fanyang graphreinforcementlearningbaseddecisionmakingtechnologyforconnectedandautonomousvehiclesframeworkreviewandfuturetrends AT ziruili graphreinforcementlearningbaseddecisionmakingtechnologyforconnectedandautonomousvehiclesframeworkreviewandfuturetrends |