Energy-Efficient Power Allocation and User Association in Heterogeneous Networks with Deep Reinforcement Learning
This paper studies the problem of joint power allocation and user association in wireless heterogeneous networks (HetNets) with a deep reinforcement learning (DRL)-based approach. This is a challenging problem since the action space is hybrid, consisting of continuous actions (power allocation) and...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-04-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/9/4135 |
| _version_ | 1827693588557856768 |
|---|---|
| author | Chi-Kai Hsieh Kun-Lin Chan Feng-Tsun Chien |
| author_facet | Chi-Kai Hsieh Kun-Lin Chan Feng-Tsun Chien |
| author_sort | Chi-Kai Hsieh |
| collection | DOAJ |
| description | This paper studies the problem of joint power allocation and user association in wireless heterogeneous networks (HetNets) with a deep reinforcement learning (DRL)-based approach. This is a challenging problem since the action space is hybrid, consisting of continuous actions (power allocation) and discrete actions (device association). Instead of quantizing the continuous space (i.e., possible values of powers) into a set of discrete alternatives and applying traditional deep reinforcement approaches such as deep Q learning, we propose working on the hybrid space directly by using the novel parameterized deep Q-network (P-DQN) to update the learning policy and maximize the average cumulative reward. Furthermore, we incorporate the constraints of limited wireless backhaul capacity and the quality-of-service (QoS) of each user equipment (UE) into the learning process. Simulation results show that the proposed P-DQN outperforms the traditional approaches, such as the DQN and distance-based association, in terms of energy efficiency while satisfying the QoS and backhaul capacity constraints. The improvement in the energy efficiency of the proposed P-DQN on average may reach 77.6% and 140.6% over the traditional DQN and distance-based association approaches, respectively, in a HetNet with three SBS and five UEs. |
| first_indexed | 2024-03-10T11:47:37Z |
| format | Article |
| id | doaj.art-db51781b816949d0b1fb1917ffa1578c |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T11:47:37Z |
| publishDate | 2021-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-db51781b816949d0b1fb1917ffa1578c2023-11-21T18:00:14ZengMDPI AGApplied Sciences2076-34172021-04-01119413510.3390/app11094135Energy-Efficient Power Allocation and User Association in Heterogeneous Networks with Deep Reinforcement LearningChi-Kai Hsieh0Kun-Lin Chan1Feng-Tsun Chien2Institute of Electronics, National Yang Ming Chiao Tung University, Hsinchu 30010, TaiwanInstitute of Electronics, National Yang Ming Chiao Tung University, Hsinchu 30010, TaiwanInstitute of Electronics, National Yang Ming Chiao Tung University, Hsinchu 30010, TaiwanThis paper studies the problem of joint power allocation and user association in wireless heterogeneous networks (HetNets) with a deep reinforcement learning (DRL)-based approach. This is a challenging problem since the action space is hybrid, consisting of continuous actions (power allocation) and discrete actions (device association). Instead of quantizing the continuous space (i.e., possible values of powers) into a set of discrete alternatives and applying traditional deep reinforcement approaches such as deep Q learning, we propose working on the hybrid space directly by using the novel parameterized deep Q-network (P-DQN) to update the learning policy and maximize the average cumulative reward. Furthermore, we incorporate the constraints of limited wireless backhaul capacity and the quality-of-service (QoS) of each user equipment (UE) into the learning process. Simulation results show that the proposed P-DQN outperforms the traditional approaches, such as the DQN and distance-based association, in terms of energy efficiency while satisfying the QoS and backhaul capacity constraints. The improvement in the energy efficiency of the proposed P-DQN on average may reach 77.6% and 140.6% over the traditional DQN and distance-based association approaches, respectively, in a HetNet with three SBS and five UEs.https://www.mdpi.com/2076-3417/11/9/4135energy efficiencypower allocationuser clusteringreinforcement learning |
| spellingShingle | Chi-Kai Hsieh Kun-Lin Chan Feng-Tsun Chien Energy-Efficient Power Allocation and User Association in Heterogeneous Networks with Deep Reinforcement Learning Applied Sciences energy efficiency power allocation user clustering reinforcement learning |
| title | Energy-Efficient Power Allocation and User Association in Heterogeneous Networks with Deep Reinforcement Learning |
| title_full | Energy-Efficient Power Allocation and User Association in Heterogeneous Networks with Deep Reinforcement Learning |
| title_fullStr | Energy-Efficient Power Allocation and User Association in Heterogeneous Networks with Deep Reinforcement Learning |
| title_full_unstemmed | Energy-Efficient Power Allocation and User Association in Heterogeneous Networks with Deep Reinforcement Learning |
| title_short | Energy-Efficient Power Allocation and User Association in Heterogeneous Networks with Deep Reinforcement Learning |
| title_sort | energy efficient power allocation and user association in heterogeneous networks with deep reinforcement learning |
| topic | energy efficiency power allocation user clustering reinforcement learning |
| url | https://www.mdpi.com/2076-3417/11/9/4135 |
| work_keys_str_mv | AT chikaihsieh energyefficientpowerallocationanduserassociationinheterogeneousnetworkswithdeepreinforcementlearning AT kunlinchan energyefficientpowerallocationanduserassociationinheterogeneousnetworkswithdeepreinforcementlearning AT fengtsunchien energyefficientpowerallocationanduserassociationinheterogeneousnetworkswithdeepreinforcementlearning |