Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device Communication
In scenarios, like critical public safety communication networks, On-Scene Available (OSA) user equipment (UE) may be only partially connected with the network infrastructure, e.g., due to physical damages or on-purpose deactivation by the authorities. In this work, we consider multi-hop Device-to-D...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-11-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/22/6692 |
| _version_ | 1827701660015656960 |
|---|---|
| author | Muhidul Islam Khan Luca Reggiani Muhammad Mahtab Alam Yannick Le Moullec Navuday Sharma Elias Yaacoub Maurizio Magarini |
| author_facet | Muhidul Islam Khan Luca Reggiani Muhammad Mahtab Alam Yannick Le Moullec Navuday Sharma Elias Yaacoub Maurizio Magarini |
| author_sort | Muhidul Islam Khan |
| collection | DOAJ |
| description | In scenarios, like critical public safety communication networks, On-Scene Available (OSA) user equipment (UE) may be only partially connected with the network infrastructure, e.g., due to physical damages or on-purpose deactivation by the authorities. In this work, we consider multi-hop Device-to-Device (D2D) communication in a hybrid infrastructure where OSA UEs connect to each other in a seamless manner in order to disseminate critical information to a deployed command center. The challenge that we address is to simultaneously keep the OSA UEs alive as long as possible and send the critical information to a final destination (e.g., a command center) as rapidly as possible, while considering the heterogeneous characteristics of the OSA UEs. We propose a dynamic adaptation approach based on machine learning to improve a joint energy-spectral efficiency (ESE). We apply a Q-learning scheme in a hybrid fashion (partially distributed and centralized) in learner agents (distributed OSA UEs) and scheduler agents (remote radio heads or RRHs), for which the next hop selection and RRH selection algorithms are proposed. Our simulation results show that the proposed dynamic adaptation approach outperforms the baseline system by approximately <inline-formula><math display="inline"><semantics><mrow><mn>67</mn><mo>%</mo></mrow></semantics></math></inline-formula> in terms of joint energy-spectral efficiency, wherein the energy efficiency of the OSA UEs benefit from a gain of approximately <inline-formula><math display="inline"><semantics><mrow><mn>30</mn><mo>%</mo></mrow></semantics></math></inline-formula>. Finally, the results show also that our proposed framework with C-RAN reduces latency by approximately <inline-formula><math display="inline"><semantics><mrow><mn>50</mn><mo>%</mo></mrow></semantics></math></inline-formula> w.r.t. the baseline. |
| first_indexed | 2024-03-10T14:38:31Z |
| format | Article |
| id | doaj.art-003e38074e614577b8d6c6324cc12029 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T14:38:31Z |
| publishDate | 2020-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-003e38074e614577b8d6c6324cc120292023-11-20T21:57:28ZengMDPI AGSensors1424-82202020-11-012022669210.3390/s20226692Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device CommunicationMuhidul Islam Khan0Luca Reggiani1Muhammad Mahtab Alam2Yannick Le Moullec3Navuday Sharma4Elias Yaacoub5Maurizio Magarini6Thomas Johann Seebeck Department of Electronics, School of Information Technology, Tallinn University of Technology, Ehitajate tee 5, 19086 Tallinn, EstoniaDipartimento di Electtronica e Informazione, Politecnico di Milano, Via Ponzio 34/5, 20133 Milano, ItalyThomas Johann Seebeck Department of Electronics, School of Information Technology, Tallinn University of Technology, Ehitajate tee 5, 19086 Tallinn, EstoniaThomas Johann Seebeck Department of Electronics, School of Information Technology, Tallinn University of Technology, Ehitajate tee 5, 19086 Tallinn, EstoniaThomas Johann Seebeck Department of Electronics, School of Information Technology, Tallinn University of Technology, Ehitajate tee 5, 19086 Tallinn, EstoniaFaculty of Computer Studies, Arab Open University, Beirut 2058 4518, LebanonDipartimento di Electtronica e Informazione, Politecnico di Milano, Via Ponzio 34/5, 20133 Milano, ItalyIn scenarios, like critical public safety communication networks, On-Scene Available (OSA) user equipment (UE) may be only partially connected with the network infrastructure, e.g., due to physical damages or on-purpose deactivation by the authorities. In this work, we consider multi-hop Device-to-Device (D2D) communication in a hybrid infrastructure where OSA UEs connect to each other in a seamless manner in order to disseminate critical information to a deployed command center. The challenge that we address is to simultaneously keep the OSA UEs alive as long as possible and send the critical information to a final destination (e.g., a command center) as rapidly as possible, while considering the heterogeneous characteristics of the OSA UEs. We propose a dynamic adaptation approach based on machine learning to improve a joint energy-spectral efficiency (ESE). We apply a Q-learning scheme in a hybrid fashion (partially distributed and centralized) in learner agents (distributed OSA UEs) and scheduler agents (remote radio heads or RRHs), for which the next hop selection and RRH selection algorithms are proposed. Our simulation results show that the proposed dynamic adaptation approach outperforms the baseline system by approximately <inline-formula><math display="inline"><semantics><mrow><mn>67</mn><mo>%</mo></mrow></semantics></math></inline-formula> in terms of joint energy-spectral efficiency, wherein the energy efficiency of the OSA UEs benefit from a gain of approximately <inline-formula><math display="inline"><semantics><mrow><mn>30</mn><mo>%</mo></mrow></semantics></math></inline-formula>. Finally, the results show also that our proposed framework with C-RAN reduces latency by approximately <inline-formula><math display="inline"><semantics><mrow><mn>50</mn><mo>%</mo></mrow></semantics></math></inline-formula> w.r.t. the baseline.https://www.mdpi.com/1424-8220/20/22/6692joint energy-spectral efficiency (ESE)device-to-device (D2D)public safety networkspervasive public safety communicationInternet of Things (IoT) |
| spellingShingle | Muhidul Islam Khan Luca Reggiani Muhammad Mahtab Alam Yannick Le Moullec Navuday Sharma Elias Yaacoub Maurizio Magarini Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device Communication Sensors joint energy-spectral efficiency (ESE) device-to-device (D2D) public safety networks pervasive public safety communication Internet of Things (IoT) |
| title | Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device Communication |
| title_full | Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device Communication |
| title_fullStr | Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device Communication |
| title_full_unstemmed | Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device Communication |
| title_short | Q-Learning Based Joint Energy-Spectral Efficiency Optimization in Multi-Hop Device-to-Device Communication |
| title_sort | q learning based joint energy spectral efficiency optimization in multi hop device to device communication |
| topic | joint energy-spectral efficiency (ESE) device-to-device (D2D) public safety networks pervasive public safety communication Internet of Things (IoT) |
| url | https://www.mdpi.com/1424-8220/20/22/6692 |
| work_keys_str_mv | AT muhidulislamkhan qlearningbasedjointenergyspectralefficiencyoptimizationinmultihopdevicetodevicecommunication AT lucareggiani qlearningbasedjointenergyspectralefficiencyoptimizationinmultihopdevicetodevicecommunication AT muhammadmahtabalam qlearningbasedjointenergyspectralefficiencyoptimizationinmultihopdevicetodevicecommunication AT yannicklemoullec qlearningbasedjointenergyspectralefficiencyoptimizationinmultihopdevicetodevicecommunication AT navudaysharma qlearningbasedjointenergyspectralefficiencyoptimizationinmultihopdevicetodevicecommunication AT eliasyaacoub qlearningbasedjointenergyspectralefficiencyoptimizationinmultihopdevicetodevicecommunication AT mauriziomagarini qlearningbasedjointenergyspectralefficiencyoptimizationinmultihopdevicetodevicecommunication |