End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm
Changing lanes while driving requires coordinating the lateral and longitudinal controls of a vehicle, considering its running state and the surrounding environment. Although the existing rule-based automated lane-changing method is simple, it is unsuitable for unpredictable scenarios encountered in...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/18/5443 |
| _version_ | 1797552953482018816 |
|---|---|
| author | Hongyu Hu Ziyang Lu Qi Wang Chengyuan Zheng |
| author_facet | Hongyu Hu Ziyang Lu Qi Wang Chengyuan Zheng |
| author_sort | Hongyu Hu |
| collection | DOAJ |
| description | Changing lanes while driving requires coordinating the lateral and longitudinal controls of a vehicle, considering its running state and the surrounding environment. Although the existing rule-based automated lane-changing method is simple, it is unsuitable for unpredictable scenarios encountered in practice. Therefore, using a deep deterministic policy gradient (DDPG) algorithm, we propose an end-to-end method for automated lane changing based on lidar data. The distance state information of the lane boundary and the surrounding vehicles obtained by the agent in a simulation environment is denoted as the state space for an automated lane-change problem based on reinforcement learning. The steering wheel angle and longitudinal acceleration are used as the action space, and both the state and action spaces are continuous. In terms of the reward function, avoiding collision and setting different expected lane-changing distances that represent different driving styles are considered for security, and the angular velocity of the steering wheel and jerk are considered for comfort. The minimum speed limit for lane changing and the control of the agent for a quick lane change are considered for efficiency. For a one-way two-lane road, a visual simulation environment scene is constructed using Pyglet. By comparing the lane-changing process tracks of two driving styles in a simplified traffic flow scene, we study the influence of driving style on the lane-changing process and lane-changing time. Through the training and adjustment of the combined lateral and longitudinal control of autonomous vehicles with different driving styles in complex traffic scenes, the vehicles could complete a series of driving tasks while considering driving-style differences. The experimental results show that autonomous vehicles can reflect the differences in the driving styles at the time of lane change at the same speed. Under the combined lateral and longitudinal control, the autonomous vehicles exhibit good robustness to different speeds and traffic density in different road sections. Thus, autonomous vehicles trained using the proposed method can learn an automated lane-changing policy while considering safety, comfort, and efficiency. |
| first_indexed | 2024-03-10T16:08:28Z |
| format | Article |
| id | doaj.art-2ec6e3327bc346198eef3f8bf1a3a83b |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T16:08:28Z |
| publishDate | 2020-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-2ec6e3327bc346198eef3f8bf1a3a83b2023-11-20T14:41:17ZengMDPI AGSensors1424-82202020-09-012018544310.3390/s20185443End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient AlgorithmHongyu Hu0Ziyang Lu1Qi Wang2Chengyuan Zheng3State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, ChinaState Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, ChinaState Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, ChinaDesign School, Loughbrough University, London E20 3BS, UKChanging lanes while driving requires coordinating the lateral and longitudinal controls of a vehicle, considering its running state and the surrounding environment. Although the existing rule-based automated lane-changing method is simple, it is unsuitable for unpredictable scenarios encountered in practice. Therefore, using a deep deterministic policy gradient (DDPG) algorithm, we propose an end-to-end method for automated lane changing based on lidar data. The distance state information of the lane boundary and the surrounding vehicles obtained by the agent in a simulation environment is denoted as the state space for an automated lane-change problem based on reinforcement learning. The steering wheel angle and longitudinal acceleration are used as the action space, and both the state and action spaces are continuous. In terms of the reward function, avoiding collision and setting different expected lane-changing distances that represent different driving styles are considered for security, and the angular velocity of the steering wheel and jerk are considered for comfort. The minimum speed limit for lane changing and the control of the agent for a quick lane change are considered for efficiency. For a one-way two-lane road, a visual simulation environment scene is constructed using Pyglet. By comparing the lane-changing process tracks of two driving styles in a simplified traffic flow scene, we study the influence of driving style on the lane-changing process and lane-changing time. Through the training and adjustment of the combined lateral and longitudinal control of autonomous vehicles with different driving styles in complex traffic scenes, the vehicles could complete a series of driving tasks while considering driving-style differences. The experimental results show that autonomous vehicles can reflect the differences in the driving styles at the time of lane change at the same speed. Under the combined lateral and longitudinal control, the autonomous vehicles exhibit good robustness to different speeds and traffic density in different road sections. Thus, autonomous vehicles trained using the proposed method can learn an automated lane-changing policy while considering safety, comfort, and efficiency.https://www.mdpi.com/1424-8220/20/18/5443intelligent vehicleautomated lane changedriving stylereinforcement learningdeep deterministic policy gradient |
| spellingShingle | Hongyu Hu Ziyang Lu Qi Wang Chengyuan Zheng End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm Sensors intelligent vehicle automated lane change driving style reinforcement learning deep deterministic policy gradient |
| title | End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm |
| title_full | End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm |
| title_fullStr | End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm |
| title_full_unstemmed | End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm |
| title_short | End-to-End Automated Lane-Change Maneuvering Considering Driving Style Using a Deep Deterministic Policy Gradient Algorithm |
| title_sort | end to end automated lane change maneuvering considering driving style using a deep deterministic policy gradient algorithm |
| topic | intelligent vehicle automated lane change driving style reinforcement learning deep deterministic policy gradient |
| url | https://www.mdpi.com/1424-8220/20/18/5443 |
| work_keys_str_mv | AT hongyuhu endtoendautomatedlanechangemaneuveringconsideringdrivingstyleusingadeepdeterministicpolicygradientalgorithm AT ziyanglu endtoendautomatedlanechangemaneuveringconsideringdrivingstyleusingadeepdeterministicpolicygradientalgorithm AT qiwang endtoendautomatedlanechangemaneuveringconsideringdrivingstyleusingadeepdeterministicpolicygradientalgorithm AT chengyuanzheng endtoendautomatedlanechangemaneuveringconsideringdrivingstyleusingadeepdeterministicpolicygradientalgorithm |