Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning
This paper presents a solution to address the challenges of unexpected events in the operation of metro trains, which can lead to increased delays and safety risks. An improved Q-learning algorithm is proposed to reschedule train timetables via incorporating train detention and different section run...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-02-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/14/4/1552 |
| _version_ | 1797299028948418560 |
|---|---|
| author | Yushen Hu Wei Li Qin Luo |
| author_facet | Yushen Hu Wei Li Qin Luo |
| author_sort | Yushen Hu |
| collection | DOAJ |
| description | This paper presents a solution to address the challenges of unexpected events in the operation of metro trains, which can lead to increased delays and safety risks. An improved Q-learning algorithm is proposed to reschedule train timetables via incorporating train detention and different section running times as actions. To enhance computational efficiency and convergence rate, a simulated annealing dynamic factor is introduced to improve action selection strategies. Additionally, importance sampling is employed to evaluate different policies effectively. A case study of Shenzhen Metro is conducted to demonstrate the effectiveness of the proposed method. The results show that the method achieves convergence, fast computation speed, and real-time adjustment capabilities. Compared to traditional methods such as no adjustment, manual adjustment, and FIFO (First-In-First-Out), the proposed method significantly reduces the average total train delay by 54% and leads to more uniform train headways. The proposed method utilizes a limited number of variables for practical state descriptions, making it well suited for real-world applications. It also exhibits good scalability and transferability to other metro systems. |
| first_indexed | 2024-03-07T22:43:37Z |
| format | Article |
| id | doaj.art-80fc59f3553748b2bf0001000edfa8e5 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-07T22:43:37Z |
| publishDate | 2024-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-80fc59f3553748b2bf0001000edfa8e52024-02-23T15:06:22ZengMDPI AGApplied Sciences2076-34172024-02-01144155210.3390/app14041552Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-LearningYushen Hu0Wei Li1Qin Luo2College of Applied Technology, Shenzhen University, Shenzhen 518060, ChinaCollege of Urban Transportation and Logistics, Shenzhen Technology University, Shenzhen 518063, ChinaCollege of Urban Transportation and Logistics, Shenzhen Technology University, Shenzhen 518063, ChinaThis paper presents a solution to address the challenges of unexpected events in the operation of metro trains, which can lead to increased delays and safety risks. An improved Q-learning algorithm is proposed to reschedule train timetables via incorporating train detention and different section running times as actions. To enhance computational efficiency and convergence rate, a simulated annealing dynamic factor is introduced to improve action selection strategies. Additionally, importance sampling is employed to evaluate different policies effectively. A case study of Shenzhen Metro is conducted to demonstrate the effectiveness of the proposed method. The results show that the method achieves convergence, fast computation speed, and real-time adjustment capabilities. Compared to traditional methods such as no adjustment, manual adjustment, and FIFO (First-In-First-Out), the proposed method significantly reduces the average total train delay by 54% and leads to more uniform train headways. The proposed method utilizes a limited number of variables for practical state descriptions, making it well suited for real-world applications. It also exhibits good scalability and transferability to other metro systems.https://www.mdpi.com/2076-3417/14/4/1552metrotimetable reschedulingtrain adjustmentreal-timeimproved Q-learning |
| spellingShingle | Yushen Hu Wei Li Qin Luo Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning Applied Sciences metro timetable rescheduling train adjustment real-time improved Q-learning |
| title | Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning |
| title_full | Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning |
| title_fullStr | Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning |
| title_full_unstemmed | Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning |
| title_short | Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning |
| title_sort | real time adjustment method for metro systems with train delays based on improved q learning |
| topic | metro timetable rescheduling train adjustment real-time improved Q-learning |
| url | https://www.mdpi.com/2076-3417/14/4/1552 |
| work_keys_str_mv | AT yushenhu realtimeadjustmentmethodformetrosystemswithtraindelaysbasedonimprovedqlearning AT weili realtimeadjustmentmethodformetrosystemswithtraindelaysbasedonimprovedqlearning AT qinluo realtimeadjustmentmethodformetrosystemswithtraindelaysbasedonimprovedqlearning |