An SDN Controller-Based Network Slicing Scheme Using Constrained Reinforcement Learning
In order to meet the strong diversification of services that demand network flexibility that will be able to serve the dire need for transmission resources, network slicing was embraced as a plausible solution. Reinforcement learning (RL) has been applied in resource allocation (RA) problems, but ha...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9982628/ |
| _version_ | 1797974413437566976 |
|---|---|
| author | Mduduzi C. Mduduzi Hlophe Bodhaswar T. Maharaj |
| author_facet | Mduduzi C. Mduduzi Hlophe Bodhaswar T. Maharaj |
| author_sort | Mduduzi C. Mduduzi Hlophe |
| collection | DOAJ |
| description | In order to meet the strong diversification of services that demand network flexibility that will be able to serve the dire need for transmission resources, network slicing was embraced as a plausible solution. Reinforcement learning (RL) has been applied in resource allocation (RA) problems, but has not yet marked the translation from traditional optimization approaches primarily due to its inability to satisfy state constraints. The aim of this article is to address this challenge. This article proposes a logical architecture for network slicing based on software-defined networking (SDN), where an SDN controller controls the network slicing process in a centralized fashion, and manages the resource allocation (RA) process with the help of the slice manager. The considered problem jointly addresses power and channel allocation using a hybrid access mode for ultra-reliable low-latency communication (URLLC) and enhanced mobile broadband (eMBB) slices. Proper assumptions on the arrival rates, packet length distributions, as well as power and delay constraints were used to design the behavior of the reward function to realize a constrained RL approach. Here, the Bellman optimality equation was reformulated into a primal-dual optimization problem through the use of Nesterov’s smoothing technique and the Legendre-Fenchel transformation. The proposed algorithm shows favorable performance over the traditional RL strategy in attributes favoring eMBB services, i.e., the average bit rate, and significantly outperforms both baselines in attributes favoring URLLC services, i.e., average latency. Systematically, on the power-delay performance evaluation, it shows that it can adapt very well in rapidly time-varying non-Markovian environments and still successfully satisfy the delay constraints of the applications hosted on a slice. |
| first_indexed | 2024-04-11T04:19:32Z |
| format | Article |
| id | doaj.art-705f2897970642dfbb5943f7fdbea919 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T04:19:32Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-705f2897970642dfbb5943f7fdbea9192022-12-31T00:00:19ZengIEEEIEEE Access2169-35362022-01-011013484813486910.1109/ACCESS.2022.32288049982628An SDN Controller-Based Network Slicing Scheme Using Constrained Reinforcement LearningMduduzi C. Mduduzi Hlophe0Bodhaswar T. Maharaj1https://orcid.org/0000-0002-3703-3637Department of Electrical, Electronic and Computer Engineering, University of Pretoria, Pretoria, South AfricaDepartment of Electrical, Electronic and Computer Engineering, University of Pretoria, Pretoria, South AfricaIn order to meet the strong diversification of services that demand network flexibility that will be able to serve the dire need for transmission resources, network slicing was embraced as a plausible solution. Reinforcement learning (RL) has been applied in resource allocation (RA) problems, but has not yet marked the translation from traditional optimization approaches primarily due to its inability to satisfy state constraints. The aim of this article is to address this challenge. This article proposes a logical architecture for network slicing based on software-defined networking (SDN), where an SDN controller controls the network slicing process in a centralized fashion, and manages the resource allocation (RA) process with the help of the slice manager. The considered problem jointly addresses power and channel allocation using a hybrid access mode for ultra-reliable low-latency communication (URLLC) and enhanced mobile broadband (eMBB) slices. Proper assumptions on the arrival rates, packet length distributions, as well as power and delay constraints were used to design the behavior of the reward function to realize a constrained RL approach. Here, the Bellman optimality equation was reformulated into a primal-dual optimization problem through the use of Nesterov’s smoothing technique and the Legendre-Fenchel transformation. The proposed algorithm shows favorable performance over the traditional RL strategy in attributes favoring eMBB services, i.e., the average bit rate, and significantly outperforms both baselines in attributes favoring URLLC services, i.e., average latency. Systematically, on the power-delay performance evaluation, it shows that it can adapt very well in rapidly time-varying non-Markovian environments and still successfully satisfy the delay constraints of the applications hosted on a slice.https://ieeexplore.ieee.org/document/9982628/5GBellman optimalityconstrained reinforcement learningeMBBmMTCnetwork slicing |
| spellingShingle | Mduduzi C. Mduduzi Hlophe Bodhaswar T. Maharaj An SDN Controller-Based Network Slicing Scheme Using Constrained Reinforcement Learning IEEE Access 5G Bellman optimality constrained reinforcement learning eMBB mMTC network slicing |
| title | An SDN Controller-Based Network Slicing Scheme Using Constrained Reinforcement Learning |
| title_full | An SDN Controller-Based Network Slicing Scheme Using Constrained Reinforcement Learning |
| title_fullStr | An SDN Controller-Based Network Slicing Scheme Using Constrained Reinforcement Learning |
| title_full_unstemmed | An SDN Controller-Based Network Slicing Scheme Using Constrained Reinforcement Learning |
| title_short | An SDN Controller-Based Network Slicing Scheme Using Constrained Reinforcement Learning |
| title_sort | sdn controller based network slicing scheme using constrained reinforcement learning |
| topic | 5G Bellman optimality constrained reinforcement learning eMBB mMTC network slicing |
| url | https://ieeexplore.ieee.org/document/9982628/ |
| work_keys_str_mv | AT mduduzicmduduzihlophe ansdncontrollerbasednetworkslicingschemeusingconstrainedreinforcementlearning AT bodhaswartmaharaj ansdncontrollerbasednetworkslicingschemeusingconstrainedreinforcementlearning AT mduduzicmduduzihlophe sdncontrollerbasednetworkslicingschemeusingconstrainedreinforcementlearning AT bodhaswartmaharaj sdncontrollerbasednetworkslicingschemeusingconstrainedreinforcementlearning |