An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement Learning
Many studies on the application of deep reinforcement learning (DRL) in the field of traffic signal control do not fully consider the influence of vehicles approaching the intersection on traffic flow. In this paper, the convolutional block attention module (CBAM) is incorporated on the basis of the...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10477432/ |
| _version_ | 1797236572090793984 |
|---|---|
| author | Peng Wang Wenlong Ni |
| author_facet | Peng Wang Wenlong Ni |
| author_sort | Peng Wang |
| collection | DOAJ |
| description | Many studies on the application of deep reinforcement learning (DRL) in the field of traffic signal control do not fully consider the influence of vehicles approaching the intersection on traffic flow. In this paper, the convolutional block attention module (CBAM) is incorporated on the basis of the Dueling Double Deep Q Network (D3QN) method to improve the sensitivity of the model to the traffic situation, which can help the model to focus more on the distribution and dynamics of vehicles near intersections. To further improve the model performance, this paper introduces the traffic light phase variable time interval based on the original D3QN method, which helps the model to take into account the traffic requirements in all directions of the intersection. In addition, Double Deep Q Network (Double DQN) and Dueling Deep Q Network (Dueling DQN) technologies are used to further improve the performance of the model. The simulation experiments using the urban traffic simulator SUMO show that the proposed method has significant advantages over D3QN, Maximum Pressure algorithm and Fixed Timing Strategy for key indicators such as mean vehicle delay time, mean queue length and average number of stops. This shows that the method proposed in this paper has great potential in practical traffic signal control applications. |
| first_indexed | 2024-04-24T17:05:59Z |
| format | Article |
| id | doaj.art-4a6782815f284fb3bf52ddd2c7922ae6 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-24T17:05:59Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-4a6782815f284fb3bf52ddd2c7922ae62024-03-28T23:00:40ZengIEEEIEEE Access2169-35362024-01-0112442244423210.1109/ACCESS.2024.338045410477432An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement LearningPeng Wang0https://orcid.org/0009-0005-7591-8099Wenlong Ni1https://orcid.org/0000-0001-8246-0411School of Digital Industry, Jiangxi Normal University, Shangrao, ChinaSchool of Digital Industry, Jiangxi Normal University, Shangrao, ChinaMany studies on the application of deep reinforcement learning (DRL) in the field of traffic signal control do not fully consider the influence of vehicles approaching the intersection on traffic flow. In this paper, the convolutional block attention module (CBAM) is incorporated on the basis of the Dueling Double Deep Q Network (D3QN) method to improve the sensitivity of the model to the traffic situation, which can help the model to focus more on the distribution and dynamics of vehicles near intersections. To further improve the model performance, this paper introduces the traffic light phase variable time interval based on the original D3QN method, which helps the model to take into account the traffic requirements in all directions of the intersection. In addition, Double Deep Q Network (Double DQN) and Dueling Deep Q Network (Dueling DQN) technologies are used to further improve the performance of the model. The simulation experiments using the urban traffic simulator SUMO show that the proposed method has significant advantages over D3QN, Maximum Pressure algorithm and Fixed Timing Strategy for key indicators such as mean vehicle delay time, mean queue length and average number of stops. This shows that the method proposed in this paper has great potential in practical traffic signal control applications.https://ieeexplore.ieee.org/document/10477432/Attention mechanismdeep reinforcement learningdeep Q networktraffic signal control |
| spellingShingle | Peng Wang Wenlong Ni An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement Learning IEEE Access Attention mechanism deep reinforcement learning deep Q network traffic signal control |
| title | An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement Learning |
| title_full | An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement Learning |
| title_fullStr | An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement Learning |
| title_full_unstemmed | An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement Learning |
| title_short | An Enhanced Dueling Double Deep Q-Network With Convolutional Block Attention Module for Traffic Signal Optimization in Deep Reinforcement Learning |
| title_sort | enhanced dueling double deep q network with convolutional block attention module for traffic signal optimization in deep reinforcement learning |
| topic | Attention mechanism deep reinforcement learning deep Q network traffic signal control |
| url | https://ieeexplore.ieee.org/document/10477432/ |
| work_keys_str_mv | AT pengwang anenhancedduelingdoubledeepqnetworkwithconvolutionalblockattentionmodulefortrafficsignaloptimizationindeepreinforcementlearning AT wenlongni anenhancedduelingdoubledeepqnetworkwithconvolutionalblockattentionmodulefortrafficsignaloptimizationindeepreinforcementlearning AT pengwang enhancedduelingdoubledeepqnetworkwithconvolutionalblockattentionmodulefortrafficsignaloptimizationindeepreinforcementlearning AT wenlongni enhancedduelingdoubledeepqnetworkwithconvolutionalblockattentionmodulefortrafficsignaloptimizationindeepreinforcementlearning |