A Deep Q-Network-Based Algorithm for Multi-Connectivity Optimization in Heterogeneous Cellular-Networks
The use of multi-connectivity has become a useful tool to manage the traffic in heterogeneous cellular network deployments, since it allows a device to be simultaneously connected to multiple cells. The proper exploitation of this technique requires to adequately configure the traffic sent through e...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-08-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/16/6179 |
| _version_ | 1827625426019680256 |
|---|---|
| author | Juan Jesús Hernández-Carlón Jordi Pérez-Romero Oriol Sallent Irene Vilà Ferran Casadevall |
| author_facet | Juan Jesús Hernández-Carlón Jordi Pérez-Romero Oriol Sallent Irene Vilà Ferran Casadevall |
| author_sort | Juan Jesús Hernández-Carlón |
| collection | DOAJ |
| description | The use of multi-connectivity has become a useful tool to manage the traffic in heterogeneous cellular network deployments, since it allows a device to be simultaneously connected to multiple cells. The proper exploitation of this technique requires to adequately configure the traffic sent through each cell depending on the experienced conditions. This motivates this work, which tackles the problem of how to optimally split the traffic among the cells when the multi-connectivity feature is used. To this end, the paper proposes the use of a deep reinforcement learning solution based on a Deep Q-Network (DQN) in order to determine the amount of traffic of a device that needs to be delivered through each cell, making the decision as a function of the current traffic and radio conditions. The obtained results show a near-optimal performance of the DQN-based solution with an average difference of only 3.9% in terms of reward with respect to the optimum strategy. Moreover, the solution clearly outperforms a reference scheme based on Signal to Interference Noise Ratio (SINR) with differences of up to 50% in terms of reward and up to 166% in terms of throughput for certain situations. Overall, the presented results show the promising performance of the DQN-based approach that establishes a basis for further research in the topic of multi-connectivity and for the application of this type of techniques in other problems of the radio access network. |
| first_indexed | 2024-03-09T12:36:37Z |
| format | Article |
| id | doaj.art-195c20cd08044553894c3b2d8d415d51 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T12:36:37Z |
| publishDate | 2022-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-195c20cd08044553894c3b2d8d415d512023-11-30T22:23:37ZengMDPI AGSensors1424-82202022-08-012216617910.3390/s22166179A Deep Q-Network-Based Algorithm for Multi-Connectivity Optimization in Heterogeneous Cellular-NetworksJuan Jesús Hernández-Carlón0Jordi Pérez-Romero1Oriol Sallent2Irene Vilà3Ferran Casadevall4Signal Theory and Communications Department, Universitat Politècnica de Catalunya (UPC), 08034 Barcelona, SpainSignal Theory and Communications Department, Universitat Politècnica de Catalunya (UPC), 08034 Barcelona, SpainSignal Theory and Communications Department, Universitat Politècnica de Catalunya (UPC), 08034 Barcelona, SpainSignal Theory and Communications Department, Universitat Politècnica de Catalunya (UPC), 08034 Barcelona, SpainSignal Theory and Communications Department, Universitat Politècnica de Catalunya (UPC), 08034 Barcelona, SpainThe use of multi-connectivity has become a useful tool to manage the traffic in heterogeneous cellular network deployments, since it allows a device to be simultaneously connected to multiple cells. The proper exploitation of this technique requires to adequately configure the traffic sent through each cell depending on the experienced conditions. This motivates this work, which tackles the problem of how to optimally split the traffic among the cells when the multi-connectivity feature is used. To this end, the paper proposes the use of a deep reinforcement learning solution based on a Deep Q-Network (DQN) in order to determine the amount of traffic of a device that needs to be delivered through each cell, making the decision as a function of the current traffic and radio conditions. The obtained results show a near-optimal performance of the DQN-based solution with an average difference of only 3.9% in terms of reward with respect to the optimum strategy. Moreover, the solution clearly outperforms a reference scheme based on Signal to Interference Noise Ratio (SINR) with differences of up to 50% in terms of reward and up to 166% in terms of throughput for certain situations. Overall, the presented results show the promising performance of the DQN-based approach that establishes a basis for further research in the topic of multi-connectivity and for the application of this type of techniques in other problems of the radio access network.https://www.mdpi.com/1424-8220/22/16/6179multi-connectivitydeep learningDeep Q-Networkheterogeneous networkscellular networks5G NR |
| spellingShingle | Juan Jesús Hernández-Carlón Jordi Pérez-Romero Oriol Sallent Irene Vilà Ferran Casadevall A Deep Q-Network-Based Algorithm for Multi-Connectivity Optimization in Heterogeneous Cellular-Networks Sensors multi-connectivity deep learning Deep Q-Network heterogeneous networks cellular networks 5G NR |
| title | A Deep Q-Network-Based Algorithm for Multi-Connectivity Optimization in Heterogeneous Cellular-Networks |
| title_full | A Deep Q-Network-Based Algorithm for Multi-Connectivity Optimization in Heterogeneous Cellular-Networks |
| title_fullStr | A Deep Q-Network-Based Algorithm for Multi-Connectivity Optimization in Heterogeneous Cellular-Networks |
| title_full_unstemmed | A Deep Q-Network-Based Algorithm for Multi-Connectivity Optimization in Heterogeneous Cellular-Networks |
| title_short | A Deep Q-Network-Based Algorithm for Multi-Connectivity Optimization in Heterogeneous Cellular-Networks |
| title_sort | deep q network based algorithm for multi connectivity optimization in heterogeneous cellular networks |
| topic | multi-connectivity deep learning Deep Q-Network heterogeneous networks cellular networks 5G NR |
| url | https://www.mdpi.com/1424-8220/22/16/6179 |
| work_keys_str_mv | AT juanjesushernandezcarlon adeepqnetworkbasedalgorithmformulticonnectivityoptimizationinheterogeneouscellularnetworks AT jordiperezromero adeepqnetworkbasedalgorithmformulticonnectivityoptimizationinheterogeneouscellularnetworks AT oriolsallent adeepqnetworkbasedalgorithmformulticonnectivityoptimizationinheterogeneouscellularnetworks AT irenevila adeepqnetworkbasedalgorithmformulticonnectivityoptimizationinheterogeneouscellularnetworks AT ferrancasadevall adeepqnetworkbasedalgorithmformulticonnectivityoptimizationinheterogeneouscellularnetworks AT juanjesushernandezcarlon deepqnetworkbasedalgorithmformulticonnectivityoptimizationinheterogeneouscellularnetworks AT jordiperezromero deepqnetworkbasedalgorithmformulticonnectivityoptimizationinheterogeneouscellularnetworks AT oriolsallent deepqnetworkbasedalgorithmformulticonnectivityoptimizationinheterogeneouscellularnetworks AT irenevila deepqnetworkbasedalgorithmformulticonnectivityoptimizationinheterogeneouscellularnetworks AT ferrancasadevall deepqnetworkbasedalgorithmformulticonnectivityoptimizationinheterogeneouscellularnetworks |