Policy Optimization of the Power Allocation Algorithm Based on the Actor–Critic Framework in Small Cell Networks
A practical solution to the power allocation problem in ultra-dense small cell networks can be achieved by using deep reinforcement learning (DRL) methods. Unlike traditional algorithms, DRL methods are capable of achieving low latency and operating without the need for global real-time channel stat...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-04-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/7/1702 |
| _version_ | 1797607445136146432 |
|---|---|
| author | Haibo Chen Zhongwei Huang Xiaorong Zhao Xiao Liu Youjun Jiang Pinyong Geng Guang Yang Yewen Cao Deqiang Wang |
| author_facet | Haibo Chen Zhongwei Huang Xiaorong Zhao Xiao Liu Youjun Jiang Pinyong Geng Guang Yang Yewen Cao Deqiang Wang |
| author_sort | Haibo Chen |
| collection | DOAJ |
| description | A practical solution to the power allocation problem in ultra-dense small cell networks can be achieved by using deep reinforcement learning (DRL) methods. Unlike traditional algorithms, DRL methods are capable of achieving low latency and operating without the need for global real-time channel state information (CSI). Based on the actor–critic framework, we propose a policy optimization of the power allocation algorithm (POPA) for small cell networks in this paper. The POPA adopts the proximal policy optimization (PPO) algorithm to update the policy, which has been shown to have stable exploration and convergence effects in our simulations. Thanks to our proposed actor–critic architecture with distributed execution and centralized exploration training, the POPA can meet real-time requirements and has multi-dimensional scalability. Through simulations, we demonstrate that the POPA outperforms existing methods in terms of spectral efficiency. Our findings suggest that the POPA can be of practical value for power allocation in small cell networks. |
| first_indexed | 2024-03-11T05:31:10Z |
| format | Article |
| id | doaj.art-d00853c5726c4bfe857d6ddc32a307d6 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-11T05:31:10Z |
| publishDate | 2023-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-d00853c5726c4bfe857d6ddc32a307d62023-11-17T17:09:27ZengMDPI AGMathematics2227-73902023-04-01117170210.3390/math11071702Policy Optimization of the Power Allocation Algorithm Based on the Actor–Critic Framework in Small Cell NetworksHaibo Chen0Zhongwei Huang1Xiaorong Zhao2Xiao Liu3Youjun Jiang4Pinyong Geng5Guang Yang6Yewen Cao7Deqiang Wang8School of Information Science and Engineering, Shandong University, Qingdao 266237, ChinaSchool of Information Science and Engineering, Shandong University, Qingdao 266237, ChinaSchool of Information Science and Engineering, Shandong University, Qingdao 266237, ChinaSchool of Information Science and Engineering, Shandong University, Qingdao 266237, ChinaSchool of Information Science and Engineering, Shandong University, Qingdao 266237, ChinaSchool of Information Science and Engineering, Shandong University, Qingdao 266237, ChinaCollege of Electronic and Information Engineering, Shandong University of Science and Technology, Qingdao 266590, ChinaSchool of Information Science and Engineering, Shandong University, Qingdao 266237, ChinaSchool of Information Science and Engineering, Shandong University, Qingdao 266237, ChinaA practical solution to the power allocation problem in ultra-dense small cell networks can be achieved by using deep reinforcement learning (DRL) methods. Unlike traditional algorithms, DRL methods are capable of achieving low latency and operating without the need for global real-time channel state information (CSI). Based on the actor–critic framework, we propose a policy optimization of the power allocation algorithm (POPA) for small cell networks in this paper. The POPA adopts the proximal policy optimization (PPO) algorithm to update the policy, which has been shown to have stable exploration and convergence effects in our simulations. Thanks to our proposed actor–critic architecture with distributed execution and centralized exploration training, the POPA can meet real-time requirements and has multi-dimensional scalability. Through simulations, we demonstrate that the POPA outperforms existing methods in terms of spectral efficiency. Our findings suggest that the POPA can be of practical value for power allocation in small cell networks.https://www.mdpi.com/2227-7390/11/7/1702power allocationdeep reinforcement learningactor–critic |
| spellingShingle | Haibo Chen Zhongwei Huang Xiaorong Zhao Xiao Liu Youjun Jiang Pinyong Geng Guang Yang Yewen Cao Deqiang Wang Policy Optimization of the Power Allocation Algorithm Based on the Actor–Critic Framework in Small Cell Networks Mathematics power allocation deep reinforcement learning actor–critic |
| title | Policy Optimization of the Power Allocation Algorithm Based on the Actor–Critic Framework in Small Cell Networks |
| title_full | Policy Optimization of the Power Allocation Algorithm Based on the Actor–Critic Framework in Small Cell Networks |
| title_fullStr | Policy Optimization of the Power Allocation Algorithm Based on the Actor–Critic Framework in Small Cell Networks |
| title_full_unstemmed | Policy Optimization of the Power Allocation Algorithm Based on the Actor–Critic Framework in Small Cell Networks |
| title_short | Policy Optimization of the Power Allocation Algorithm Based on the Actor–Critic Framework in Small Cell Networks |
| title_sort | policy optimization of the power allocation algorithm based on the actor critic framework in small cell networks |
| topic | power allocation deep reinforcement learning actor–critic |
| url | https://www.mdpi.com/2227-7390/11/7/1702 |
| work_keys_str_mv | AT haibochen policyoptimizationofthepowerallocationalgorithmbasedontheactorcriticframeworkinsmallcellnetworks AT zhongweihuang policyoptimizationofthepowerallocationalgorithmbasedontheactorcriticframeworkinsmallcellnetworks AT xiaorongzhao policyoptimizationofthepowerallocationalgorithmbasedontheactorcriticframeworkinsmallcellnetworks AT xiaoliu policyoptimizationofthepowerallocationalgorithmbasedontheactorcriticframeworkinsmallcellnetworks AT youjunjiang policyoptimizationofthepowerallocationalgorithmbasedontheactorcriticframeworkinsmallcellnetworks AT pinyonggeng policyoptimizationofthepowerallocationalgorithmbasedontheactorcriticframeworkinsmallcellnetworks AT guangyang policyoptimizationofthepowerallocationalgorithmbasedontheactorcriticframeworkinsmallcellnetworks AT yewencao policyoptimizationofthepowerallocationalgorithmbasedontheactorcriticframeworkinsmallcellnetworks AT deqiangwang policyoptimizationofthepowerallocationalgorithmbasedontheactorcriticframeworkinsmallcellnetworks |