On Meeting a Maximum Delay Constraint Using Reinforcement Learning
Several emerging applications in wireless communications are required to achieve low latency, but also high traffic rates and reliabilities. From a latency point of view, most of the state-of-the-art techniques consider the average latency which may not directly apply to scenarios with stringent lat...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9887962/ |
| _version_ | 1798000468395294720 |
|---|---|
| author | Hossein Shafieirad Raviraj S. Adve |
| author_facet | Hossein Shafieirad Raviraj S. Adve |
| author_sort | Hossein Shafieirad |
| collection | DOAJ |
| description | Several emerging applications in wireless communications are required to achieve low latency, but also high traffic rates and reliabilities. From a latency point of view, most of the state-of-the-art techniques consider the average latency which may not directly apply to scenarios with stringent latency constraints. In this paper, we consider scheduling under a max-delay constraint; this is an NP-hard problem. We propose a novel approach to tackle the scheduling problem by directly addressing the constraint. We consider the downlink of a multi-cell wireless communication network with nodes communicating with users each facing their own delay constraint on randomly arrived packets. Packets must be scheduled to meet the users’ delay constraints. Our main contributions are first, proposing a new search approach, Super State Monte-Carlo Tree Search (SS-MCTS), as a version of regular MCTS modified for large-scale probabilistic environments; second, developing trained value and policy networks to reduce computational complexity, and finally, addressing the scheduling problem through a reinforcement learning framework. Our numerical results demonstrate that the proposed approach significantly improves the packet delivery rate over a baseline approach while meeting the max-delay constraint, and addressing the scalability as the main issues in large action-state spaces. |
| first_indexed | 2024-04-11T11:20:45Z |
| format | Article |
| id | doaj.art-9877bc201e44404b8aac2d8c968e386a |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T11:20:45Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-9877bc201e44404b8aac2d8c968e386a2022-12-22T04:27:04ZengIEEEIEEE Access2169-35362022-01-0110978979791110.1109/ACCESS.2022.32060359887962On Meeting a Maximum Delay Constraint Using Reinforcement LearningHossein Shafieirad0https://orcid.org/0000-0002-0633-404XRaviraj S. Adve1https://orcid.org/0000-0003-0224-2209Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, CanadaDepartment of Electrical and Computer Engineering, University of Toronto, Toronto, ON, CanadaSeveral emerging applications in wireless communications are required to achieve low latency, but also high traffic rates and reliabilities. From a latency point of view, most of the state-of-the-art techniques consider the average latency which may not directly apply to scenarios with stringent latency constraints. In this paper, we consider scheduling under a max-delay constraint; this is an NP-hard problem. We propose a novel approach to tackle the scheduling problem by directly addressing the constraint. We consider the downlink of a multi-cell wireless communication network with nodes communicating with users each facing their own delay constraint on randomly arrived packets. Packets must be scheduled to meet the users’ delay constraints. Our main contributions are first, proposing a new search approach, Super State Monte-Carlo Tree Search (SS-MCTS), as a version of regular MCTS modified for large-scale probabilistic environments; second, developing trained value and policy networks to reduce computational complexity, and finally, addressing the scheduling problem through a reinforcement learning framework. Our numerical results demonstrate that the proposed approach significantly improves the packet delivery rate over a baseline approach while meeting the max-delay constraint, and addressing the scalability as the main issues in large action-state spaces.https://ieeexplore.ieee.org/document/9887962/Monte-Carlo tree searchschedulingreinforcement learningmax-delay constraints |
| spellingShingle | Hossein Shafieirad Raviraj S. Adve On Meeting a Maximum Delay Constraint Using Reinforcement Learning IEEE Access Monte-Carlo tree search scheduling reinforcement learning max-delay constraints |
| title | On Meeting a Maximum Delay Constraint Using Reinforcement Learning |
| title_full | On Meeting a Maximum Delay Constraint Using Reinforcement Learning |
| title_fullStr | On Meeting a Maximum Delay Constraint Using Reinforcement Learning |
| title_full_unstemmed | On Meeting a Maximum Delay Constraint Using Reinforcement Learning |
| title_short | On Meeting a Maximum Delay Constraint Using Reinforcement Learning |
| title_sort | on meeting a maximum delay constraint using reinforcement learning |
| topic | Monte-Carlo tree search scheduling reinforcement learning max-delay constraints |
| url | https://ieeexplore.ieee.org/document/9887962/ |
| work_keys_str_mv | AT hosseinshafieirad onmeetingamaximumdelayconstraintusingreinforcementlearning AT ravirajsadve onmeetingamaximumdelayconstraintusingreinforcementlearning |