Learning to Drive in the NGSIM Simulator Using Proximal Policy Optimization
As a popular research field, autonomous driving may offer great benefits for human society. To achieve that, current studies often applied machine learning methods like reinforcement learning to enable an agent to interact and learn in a stimulating environment. However, most simulators lack realist...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Hindawi-Wiley
2023-01-01
|
| Series: | Journal of Advanced Transportation |
| Online Access: | http://dx.doi.org/10.1155/2023/4127486 |
| _version_ | 1797849324318621696 |
|---|---|
| author | Yang Zhou Yunxing Chen |
| author_facet | Yang Zhou Yunxing Chen |
| author_sort | Yang Zhou |
| collection | DOAJ |
| description | As a popular research field, autonomous driving may offer great benefits for human society. To achieve that, current studies often applied machine learning methods like reinforcement learning to enable an agent to interact and learn in a stimulating environment. However, most simulators lack realistic traffic which may cause a deficiency in realistic interaction. The present study adopted the SMARTS platform to create a simulator in which the trajectories of the vehicles in the NGSIM I-80 dataset were extracted as the background traffic. The built NGSIM simulator was used to train a model using the proximal policy optimization method. The actor-critic neural network was applied, and the model takes inputs including 38 features that encode the information of the host vehicle and the nearest surrounding vehicles in the current lane and adjacent lane. A2C was selected as a comparative method. The results revealed that the PPO model outperformed the A2C model in the current task by collecting more rewards, traveling longer distances, and encountering less dangerous events during model training and testing. The PPO model achieved an 84% success rate in the test which is comparable to the related studies. The present study proved that the public driving dataset and reinforcement learning can provide a useful tool to achieve autonomous driving. |
| first_indexed | 2024-04-09T18:42:06Z |
| format | Article |
| id | doaj.art-acb32121123441128189959f9c7ef1f9 |
| institution | Directory Open Access Journal |
| issn | 2042-3195 |
| language | English |
| last_indexed | 2024-04-09T18:42:06Z |
| publishDate | 2023-01-01 |
| publisher | Hindawi-Wiley |
| record_format | Article |
| series | Journal of Advanced Transportation |
| spelling | doaj.art-acb32121123441128189959f9c7ef1f92023-04-11T00:00:54ZengHindawi-WileyJournal of Advanced Transportation2042-31952023-01-01202310.1155/2023/4127486Learning to Drive in the NGSIM Simulator Using Proximal Policy OptimizationYang Zhou0Yunxing Chen1School of Vehicle EngineeringHubei Key Laboratory of Power System Design and Test for Electrical VehicleAs a popular research field, autonomous driving may offer great benefits for human society. To achieve that, current studies often applied machine learning methods like reinforcement learning to enable an agent to interact and learn in a stimulating environment. However, most simulators lack realistic traffic which may cause a deficiency in realistic interaction. The present study adopted the SMARTS platform to create a simulator in which the trajectories of the vehicles in the NGSIM I-80 dataset were extracted as the background traffic. The built NGSIM simulator was used to train a model using the proximal policy optimization method. The actor-critic neural network was applied, and the model takes inputs including 38 features that encode the information of the host vehicle and the nearest surrounding vehicles in the current lane and adjacent lane. A2C was selected as a comparative method. The results revealed that the PPO model outperformed the A2C model in the current task by collecting more rewards, traveling longer distances, and encountering less dangerous events during model training and testing. The PPO model achieved an 84% success rate in the test which is comparable to the related studies. The present study proved that the public driving dataset and reinforcement learning can provide a useful tool to achieve autonomous driving.http://dx.doi.org/10.1155/2023/4127486 |
| spellingShingle | Yang Zhou Yunxing Chen Learning to Drive in the NGSIM Simulator Using Proximal Policy Optimization Journal of Advanced Transportation |
| title | Learning to Drive in the NGSIM Simulator Using Proximal Policy Optimization |
| title_full | Learning to Drive in the NGSIM Simulator Using Proximal Policy Optimization |
| title_fullStr | Learning to Drive in the NGSIM Simulator Using Proximal Policy Optimization |
| title_full_unstemmed | Learning to Drive in the NGSIM Simulator Using Proximal Policy Optimization |
| title_short | Learning to Drive in the NGSIM Simulator Using Proximal Policy Optimization |
| title_sort | learning to drive in the ngsim simulator using proximal policy optimization |
| url | http://dx.doi.org/10.1155/2023/4127486 |
| work_keys_str_mv | AT yangzhou learningtodriveinthengsimsimulatorusingproximalpolicyoptimization AT yunxingchen learningtodriveinthengsimsimulatorusingproximalpolicyoptimization |