Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network
Cloud radio access network (CRAN) has been shown as an effective means to boost network performance. Such gain stems from the intelligent management of remote radio heads (RRHs) in terms of on/off operation mode and power consumption. Most conventional resource allocation (RA) methods, however, opti...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9336582/ |
| _version_ | 1818428847340650496 |
|---|---|
| author | Amjad Iqbal Mau-Luen Tham Yoong Choon Chang |
| author_facet | Amjad Iqbal Mau-Luen Tham Yoong Choon Chang |
| author_sort | Amjad Iqbal |
| collection | DOAJ |
| description | Cloud radio access network (CRAN) has been shown as an effective means to boost network performance. Such gain stems from the intelligent management of remote radio heads (RRHs) in terms of on/off operation mode and power consumption. Most conventional resource allocation (RA) methods, however, optimize the network utility without considering the switching overhead of RRHs in adjacent time intervals. When the network environment becomes time-correlated, mathematical optimization is not directly applicable. In this paper, we aim to optimize the energy efficiency (EE) subject to the constraints on per-RRH transmission power and user data rates. To this end, we formulate the EE problem as a Markov decision process (MDP) and subsequently adopt deep reinforcement learning (DRL) technique to reap the cumulative EE rewards. Our starting point is the deep Q network (DQN), which is a combination of deep learning and Q-learning. In each time slot, DQN configures the status of RRHs yielding the largest Q-value (known as state-action value) prior to solving a power minimization problem for active RRHs. To overcome the Q-value overestimation issue of DQN, we propose a Double DQN (DDQN) framework that obtains optimal reward better than DQN by separating the selected action from the target Q-value generator. Simulation results validate that the DDQN-based RA method is more energy-efficient than the DQN-based RA algorithm and a baseline solution. |
| first_indexed | 2024-12-14T15:08:07Z |
| format | Article |
| id | doaj.art-a48fdc60a2724cb69c6259aa62ff615b |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-14T15:08:07Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-a48fdc60a2724cb69c6259aa62ff615b2022-12-21T22:56:40ZengIEEEIEEE Access2169-35362021-01-019204402044910.1109/ACCESS.2021.30549099336582Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access NetworkAmjad Iqbal0https://orcid.org/0000-0002-3720-2592Mau-Luen Tham1Yoong Choon Chang2Department of Electrical and Electronic Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Sungai Long Campus, Selangor, MalaysiaDepartment of Electrical and Electronic Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Sungai Long Campus, Selangor, MalaysiaDepartment of Electrical and Electronic Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Sungai Long Campus, Selangor, MalaysiaCloud radio access network (CRAN) has been shown as an effective means to boost network performance. Such gain stems from the intelligent management of remote radio heads (RRHs) in terms of on/off operation mode and power consumption. Most conventional resource allocation (RA) methods, however, optimize the network utility without considering the switching overhead of RRHs in adjacent time intervals. When the network environment becomes time-correlated, mathematical optimization is not directly applicable. In this paper, we aim to optimize the energy efficiency (EE) subject to the constraints on per-RRH transmission power and user data rates. To this end, we formulate the EE problem as a Markov decision process (MDP) and subsequently adopt deep reinforcement learning (DRL) technique to reap the cumulative EE rewards. Our starting point is the deep Q network (DQN), which is a combination of deep learning and Q-learning. In each time slot, DQN configures the status of RRHs yielding the largest Q-value (known as state-action value) prior to solving a power minimization problem for active RRHs. To overcome the Q-value overestimation issue of DQN, we propose a Double DQN (DDQN) framework that obtains optimal reward better than DQN by separating the selected action from the target Q-value generator. Simulation results validate that the DDQN-based RA method is more energy-efficient than the DQN-based RA algorithm and a baseline solution.https://ieeexplore.ieee.org/document/9336582/Cloud RANdouble deep Q-network (DDQN)energy efficiency (EE)Markov decision process (MDP)power allocation |
| spellingShingle | Amjad Iqbal Mau-Luen Tham Yoong Choon Chang Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network IEEE Access Cloud RAN double deep Q-network (DDQN) energy efficiency (EE) Markov decision process (MDP) power allocation |
| title | Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network |
| title_full | Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network |
| title_fullStr | Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network |
| title_full_unstemmed | Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network |
| title_short | Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network |
| title_sort | double deep q network based energy efficient resource allocation in cloud radio access network |
| topic | Cloud RAN double deep Q-network (DDQN) energy efficiency (EE) Markov decision process (MDP) power allocation |
| url | https://ieeexplore.ieee.org/document/9336582/ |
| work_keys_str_mv | AT amjadiqbal doubledeepqnetworkbasedenergyefficientresourceallocationincloudradioaccessnetwork AT mauluentham doubledeepqnetworkbasedenergyefficientresourceallocationincloudradioaccessnetwork AT yoongchoonchang doubledeepqnetworkbasedenergyefficientresourceallocationincloudradioaccessnetwork |