Robust Reinforcement Learning Strategies with Evolving Curriculum for Efficient Bus Operations in Smart Cities
Public transit systems are critical to the quality of urban life, and enhancing their efficiency is essential for building cost-effective and sustainable smart cities. Historically, researchers sought reinforcement learning (RL) applications to mitigate bus bunching issues with holding strategies. N...
| Main Authors: | , , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Multidisciplinary Digital Publishing Institute
2025
|
| Online Access: | https://hdl.handle.net/1721.1/157936 |
| _version_ | 1824458172017934336 |
|---|---|
| author | Tang, Yuhan Qu, Ao Jiang, Xuan Mo, Baichuan Cao, Shangqing Rodriguez, Joseph Koutsopoulos, Haris N Wu, Cathy Zhao, Jinhua |
| author2 | Massachusetts Institute of Technology. Department of Civil and Environmental Engineering |
| author_facet | Massachusetts Institute of Technology. Department of Civil and Environmental Engineering Tang, Yuhan Qu, Ao Jiang, Xuan Mo, Baichuan Cao, Shangqing Rodriguez, Joseph Koutsopoulos, Haris N Wu, Cathy Zhao, Jinhua |
| author_sort | Tang, Yuhan |
| collection | MIT |
| description | Public transit systems are critical to the quality of urban life, and enhancing their efficiency is essential for building cost-effective and sustainable smart cities. Historically, researchers sought reinforcement learning (RL) applications to mitigate bus bunching issues with holding strategies. Nonetheless, these attempts often led to oversimplifications and misalignment with the goal of reducing the total time passengers spent in the system, resulting in less robust or non-optimal solutions. In this study, we introduce a novel setting where each bus, supervised by an RL agent, can appropriately form aggregated policies from three strategies (holding, skipping station, and turning around to serve the opposite direction). It’s difficult to learn them all together, due to learning complexity, we employ domain knowledge and develop a gradually expanding action space curriculum, enabling agents to learn these strategies incrementally. We incorporate Long Short-Term Memory (LSTM) in our model considering the temporal interrelation among these actions. To address the inherent uncertainties of real-world traffic systems, we impose Domain Randomization (DR) on variables such as passenger demand and bus schedules. We conduct extensive numerical experiments with the integration of synthetic and real-world data to evaluate our model. Our methodology proves effective, enhancing bus schedule reliability and reducing total passenger waiting time by over 15%, thereby improving bus operation efficiency and smoothering operations of buses that align with sustainable goals. This work highlights the potential of robust RL combined with curriculum learning for optimizing public transport in smart cities, offering a scalable solution for real-world multi-agent systems. |
| first_indexed | 2025-02-19T04:21:39Z |
| format | Article |
| id | mit-1721.1/157936 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2025-02-19T04:21:39Z |
| publishDate | 2025 |
| publisher | Multidisciplinary Digital Publishing Institute |
| record_format | dspace |
| spelling | mit-1721.1/1579362025-02-14T16:04:04Z Robust Reinforcement Learning Strategies with Evolving Curriculum for Efficient Bus Operations in Smart Cities Tang, Yuhan Qu, Ao Jiang, Xuan Mo, Baichuan Cao, Shangqing Rodriguez, Joseph Koutsopoulos, Haris N Wu, Cathy Zhao, Jinhua Massachusetts Institute of Technology. Department of Civil and Environmental Engineering Massachusetts Institute of Technology. Department of Urban Studies and Planning Public transit systems are critical to the quality of urban life, and enhancing their efficiency is essential for building cost-effective and sustainable smart cities. Historically, researchers sought reinforcement learning (RL) applications to mitigate bus bunching issues with holding strategies. Nonetheless, these attempts often led to oversimplifications and misalignment with the goal of reducing the total time passengers spent in the system, resulting in less robust or non-optimal solutions. In this study, we introduce a novel setting where each bus, supervised by an RL agent, can appropriately form aggregated policies from three strategies (holding, skipping station, and turning around to serve the opposite direction). It’s difficult to learn them all together, due to learning complexity, we employ domain knowledge and develop a gradually expanding action space curriculum, enabling agents to learn these strategies incrementally. We incorporate Long Short-Term Memory (LSTM) in our model considering the temporal interrelation among these actions. To address the inherent uncertainties of real-world traffic systems, we impose Domain Randomization (DR) on variables such as passenger demand and bus schedules. We conduct extensive numerical experiments with the integration of synthetic and real-world data to evaluate our model. Our methodology proves effective, enhancing bus schedule reliability and reducing total passenger waiting time by over 15%, thereby improving bus operation efficiency and smoothering operations of buses that align with sustainable goals. This work highlights the potential of robust RL combined with curriculum learning for optimizing public transport in smart cities, offering a scalable solution for real-world multi-agent systems. 2025-01-02T17:50:53Z 2025-01-02T17:50:53Z 2024-11-29 2024-12-27T14:02:37Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/157936 Tang, Y.; Qu, A.; Jiang, X.; Mo, B.; Cao, S.; Rodriguez, J.; Koutsopoulos, H.N.; Wu, C.; Zhao, J. Robust Reinforcement Learning Strategies with Evolving Curriculum for Efficient Bus Operations in Smart Cities. Smart Cities 2024, 7, 3658-3677. PUBLISHER_CC http://dx.doi.org/10.3390/smartcities7060141 Smart Cities Creative Commons Attribution https://creativecommons.org/licenses/by/4.0/ application/pdf Multidisciplinary Digital Publishing Institute Multidisciplinary Digital Publishing Institute |
| spellingShingle | Tang, Yuhan Qu, Ao Jiang, Xuan Mo, Baichuan Cao, Shangqing Rodriguez, Joseph Koutsopoulos, Haris N Wu, Cathy Zhao, Jinhua Robust Reinforcement Learning Strategies with Evolving Curriculum for Efficient Bus Operations in Smart Cities |
| title | Robust Reinforcement Learning Strategies with Evolving Curriculum for Efficient Bus Operations in Smart Cities |
| title_full | Robust Reinforcement Learning Strategies with Evolving Curriculum for Efficient Bus Operations in Smart Cities |
| title_fullStr | Robust Reinforcement Learning Strategies with Evolving Curriculum for Efficient Bus Operations in Smart Cities |
| title_full_unstemmed | Robust Reinforcement Learning Strategies with Evolving Curriculum for Efficient Bus Operations in Smart Cities |
| title_short | Robust Reinforcement Learning Strategies with Evolving Curriculum for Efficient Bus Operations in Smart Cities |
| title_sort | robust reinforcement learning strategies with evolving curriculum for efficient bus operations in smart cities |
| url | https://hdl.handle.net/1721.1/157936 |
| work_keys_str_mv | AT tangyuhan robustreinforcementlearningstrategieswithevolvingcurriculumforefficientbusoperationsinsmartcities AT quao robustreinforcementlearningstrategieswithevolvingcurriculumforefficientbusoperationsinsmartcities AT jiangxuan robustreinforcementlearningstrategieswithevolvingcurriculumforefficientbusoperationsinsmartcities AT mobaichuan robustreinforcementlearningstrategieswithevolvingcurriculumforefficientbusoperationsinsmartcities AT caoshangqing robustreinforcementlearningstrategieswithevolvingcurriculumforefficientbusoperationsinsmartcities AT rodriguezjoseph robustreinforcementlearningstrategieswithevolvingcurriculumforefficientbusoperationsinsmartcities AT koutsopoulosharisn robustreinforcementlearningstrategieswithevolvingcurriculumforefficientbusoperationsinsmartcities AT wucathy robustreinforcementlearningstrategieswithevolvingcurriculumforefficientbusoperationsinsmartcities AT zhaojinhua robustreinforcementlearningstrategieswithevolvingcurriculumforefficientbusoperationsinsmartcities |