Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning
The optimization and control of traffic signals is very important for logistics transportation. It not only improves the operational efficiency and safety of road traffic, but also conforms to the direction of the intelligent, green, and sustainable development of modern cities. In order to improve...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-01-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/13/1/198 |
| _version_ | 1797358948515315712 |
|---|---|
| author | Kerang Cao Liwei Wang Shuo Zhang Lini Duan Guimin Jiang Stefano Sfarra Hai Zhang Hoekyung Jung |
| author_facet | Kerang Cao Liwei Wang Shuo Zhang Lini Duan Guimin Jiang Stefano Sfarra Hai Zhang Hoekyung Jung |
| author_sort | Kerang Cao |
| collection | DOAJ |
| description | The optimization and control of traffic signals is very important for logistics transportation. It not only improves the operational efficiency and safety of road traffic, but also conforms to the direction of the intelligent, green, and sustainable development of modern cities. In order to improve the optimization effect of traffic signal control, this paper proposes a traffic signal optimization method based on deep reinforcement learning and Simulation of Urban Mobility (SUMO) software for urban traffic scenarios. The intersection training scenario was established using SUMO micro traffic simulation software, and the maximum vehicle queue length and vehicle queue time were selected as performance evaluation indicators. In order to be more relevant to the real environment, the experiment uses Weibull distribution to simulate vehicle generation. Since deep reinforcement learning takes into account both perceptual and decision-making capabilities, this study proposes a traffic signal optimization control model based on the deep reinforcement learning Deep Q Network (DQN) algorithm by considering the realism and complexity of traffic intersections, and first uses the DQN algorithm to train the model in a training scenario. After that, the G-DQN (Grouping-DQN) algorithm is proposed to address the problems that the definition of states in existing studies cannot accurately represent the traffic states and the slow convergence of neural networks. Finally, the performance of the G-DQN algorithm model was compared with the original DQN algorithm model and Advantage Actor-Critic (A2C) algorithm model. The experimental results show that the improved algorithm increased the main indicators in all aspects. |
| first_indexed | 2024-03-08T15:08:37Z |
| format | Article |
| id | doaj.art-9865086474ee4979b1831efecbd22570 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-08T15:08:37Z |
| publishDate | 2024-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-9865086474ee4979b1831efecbd225702024-01-10T14:55:07ZengMDPI AGElectronics2079-92922024-01-0113119810.3390/electronics13010198Optimization Control of Adaptive Traffic Signal with Deep Reinforcement LearningKerang Cao0Liwei Wang1Shuo Zhang2Lini Duan3Guimin Jiang4Stefano Sfarra5Hai Zhang6Hoekyung Jung7Key Laboratory of Intelligent Technology of Chemical Process Industry in Liaoning Province, College of Computer Science and Technology, Shenyang University of Chemical Technology, Shenyang 110142, ChinaKey Laboratory of Intelligent Technology of Chemical Process Industry in Liaoning Province, College of Computer Science and Technology, Shenyang University of Chemical Technology, Shenyang 110142, ChinaKey Laboratory of Intelligent Technology of Chemical Process Industry in Liaoning Province, College of Computer Science and Technology, Shenyang University of Chemical Technology, Shenyang 110142, ChinaBig Data Management and Application, Shenyang University of Chemical Technology, Shenyang 110142, ChinaSchool of Automation and Electrical Engineering, Shenyang Ligong University, Shenyang 110159, ChinaDepartment of Industrial and Information Engineering and Economics, University of L’Aquila, I-67100 L’Aquila, ItalyCentre for Composite Materials and Structures (CCMS), Harbin Institute of Technology, Harbin 150001, ChinaComputer Engineering Department, Paichai University, Daejeon 35345, Republic of KoreaThe optimization and control of traffic signals is very important for logistics transportation. It not only improves the operational efficiency and safety of road traffic, but also conforms to the direction of the intelligent, green, and sustainable development of modern cities. In order to improve the optimization effect of traffic signal control, this paper proposes a traffic signal optimization method based on deep reinforcement learning and Simulation of Urban Mobility (SUMO) software for urban traffic scenarios. The intersection training scenario was established using SUMO micro traffic simulation software, and the maximum vehicle queue length and vehicle queue time were selected as performance evaluation indicators. In order to be more relevant to the real environment, the experiment uses Weibull distribution to simulate vehicle generation. Since deep reinforcement learning takes into account both perceptual and decision-making capabilities, this study proposes a traffic signal optimization control model based on the deep reinforcement learning Deep Q Network (DQN) algorithm by considering the realism and complexity of traffic intersections, and first uses the DQN algorithm to train the model in a training scenario. After that, the G-DQN (Grouping-DQN) algorithm is proposed to address the problems that the definition of states in existing studies cannot accurately represent the traffic states and the slow convergence of neural networks. Finally, the performance of the G-DQN algorithm model was compared with the original DQN algorithm model and Advantage Actor-Critic (A2C) algorithm model. The experimental results show that the improved algorithm increased the main indicators in all aspects.https://www.mdpi.com/2079-9292/13/1/198logistics transportationtraffic signal optimization controlintelligent optimizationdeep reinforcement learningDQN |
| spellingShingle | Kerang Cao Liwei Wang Shuo Zhang Lini Duan Guimin Jiang Stefano Sfarra Hai Zhang Hoekyung Jung Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning Electronics logistics transportation traffic signal optimization control intelligent optimization deep reinforcement learning DQN |
| title | Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning |
| title_full | Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning |
| title_fullStr | Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning |
| title_full_unstemmed | Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning |
| title_short | Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning |
| title_sort | optimization control of adaptive traffic signal with deep reinforcement learning |
| topic | logistics transportation traffic signal optimization control intelligent optimization deep reinforcement learning DQN |
| url | https://www.mdpi.com/2079-9292/13/1/198 |
| work_keys_str_mv | AT kerangcao optimizationcontrolofadaptivetrafficsignalwithdeepreinforcementlearning AT liweiwang optimizationcontrolofadaptivetrafficsignalwithdeepreinforcementlearning AT shuozhang optimizationcontrolofadaptivetrafficsignalwithdeepreinforcementlearning AT liniduan optimizationcontrolofadaptivetrafficsignalwithdeepreinforcementlearning AT guiminjiang optimizationcontrolofadaptivetrafficsignalwithdeepreinforcementlearning AT stefanosfarra optimizationcontrolofadaptivetrafficsignalwithdeepreinforcementlearning AT haizhang optimizationcontrolofadaptivetrafficsignalwithdeepreinforcementlearning AT hoekyungjung optimizationcontrolofadaptivetrafficsignalwithdeepreinforcementlearning |