Constrained DRL for Energy Efficiency Optimization in RSMA-Based Integrated Satellite Terrestrial Network
To accommodate the requirements of extensive coverage and ubiquitous connectivity in 6G communications, satellite plays a more significant role in it. As users and devices explosively grow, new multiple access technologies are called for. Among the new candidates, rate splitting multiple access (RSM...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/18/7859 |
| _version_ | 1797577001863741440 |
|---|---|
| author | Qingmiao Zhang Lidong Zhu Yanyan Chen Shan Jiang |
| author_facet | Qingmiao Zhang Lidong Zhu Yanyan Chen Shan Jiang |
| author_sort | Qingmiao Zhang |
| collection | DOAJ |
| description | To accommodate the requirements of extensive coverage and ubiquitous connectivity in 6G communications, satellite plays a more significant role in it. As users and devices explosively grow, new multiple access technologies are called for. Among the new candidates, rate splitting multiple access (RSMA) shows great potential. Since satellites are power-limited, we investigate the energy-efficient resource allocation in the integrated satellite terrestrial network (ISTN)-adopting RSMA scheme in this paper. However, this non-convex problem is challenging to solve using conventional model-based methods. Because this optimization task has a quality of service (QoS) requirement and continuous action/state space, we propose to use constrained soft actor-critic (SAC) to tackle it. This policy-gradient algorithm incorporates the Lagrangian relaxation technique to convert the original constrained problem into a penalized unconstrained one. The reward is maximized while the requirements are satisfied. Moreover, the learning process is time-consuming and unnecessary when little changes in the network. So, an on–off mechanism is introduced to avoid this situation. By calculating the difference between the current state and the last one, the system will decide to learn a new action or take the last one. The simulation results show that the proposed algorithm can outperform other benchmark algorithms in terms of energy efficiency while satisfying the QoS constraint. In addition, the time consumption is lowered because of the on–off design. |
| first_indexed | 2024-03-10T22:02:44Z |
| format | Article |
| id | doaj.art-d00fe6b3173a4b9da05bbc4ca87b3efb |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T22:02:44Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-d00fe6b3173a4b9da05bbc4ca87b3efb2023-11-19T12:55:12ZengMDPI AGSensors1424-82202023-09-012318785910.3390/s23187859Constrained DRL for Energy Efficiency Optimization in RSMA-Based Integrated Satellite Terrestrial NetworkQingmiao Zhang0Lidong Zhu1Yanyan Chen2Shan Jiang3National Key Laboratory of Science and Technology on Communications, University of Electronic Science and Technology of China, Chengdu 611731, ChinaNational Key Laboratory of Science and Technology on Communications, University of Electronic Science and Technology of China, Chengdu 611731, ChinaSchool of Computer Science and Information Engineering, Xiamen Institute of Technology, Xiamen 361021, ChinaChina Mobile (Jiangxi) Communications Group Co., Ltd., Yichun 336000, ChinaTo accommodate the requirements of extensive coverage and ubiquitous connectivity in 6G communications, satellite plays a more significant role in it. As users and devices explosively grow, new multiple access technologies are called for. Among the new candidates, rate splitting multiple access (RSMA) shows great potential. Since satellites are power-limited, we investigate the energy-efficient resource allocation in the integrated satellite terrestrial network (ISTN)-adopting RSMA scheme in this paper. However, this non-convex problem is challenging to solve using conventional model-based methods. Because this optimization task has a quality of service (QoS) requirement and continuous action/state space, we propose to use constrained soft actor-critic (SAC) to tackle it. This policy-gradient algorithm incorporates the Lagrangian relaxation technique to convert the original constrained problem into a penalized unconstrained one. The reward is maximized while the requirements are satisfied. Moreover, the learning process is time-consuming and unnecessary when little changes in the network. So, an on–off mechanism is introduced to avoid this situation. By calculating the difference between the current state and the last one, the system will decide to learn a new action or take the last one. The simulation results show that the proposed algorithm can outperform other benchmark algorithms in terms of energy efficiency while satisfying the QoS constraint. In addition, the time consumption is lowered because of the on–off design.https://www.mdpi.com/1424-8220/23/18/7859integrated satellite terrestrial networkrate splitting multiple accessenergy efficiencyconstrained deep reinforcement learningsoft actor-critic |
| spellingShingle | Qingmiao Zhang Lidong Zhu Yanyan Chen Shan Jiang Constrained DRL for Energy Efficiency Optimization in RSMA-Based Integrated Satellite Terrestrial Network Sensors integrated satellite terrestrial network rate splitting multiple access energy efficiency constrained deep reinforcement learning soft actor-critic |
| title | Constrained DRL for Energy Efficiency Optimization in RSMA-Based Integrated Satellite Terrestrial Network |
| title_full | Constrained DRL for Energy Efficiency Optimization in RSMA-Based Integrated Satellite Terrestrial Network |
| title_fullStr | Constrained DRL for Energy Efficiency Optimization in RSMA-Based Integrated Satellite Terrestrial Network |
| title_full_unstemmed | Constrained DRL for Energy Efficiency Optimization in RSMA-Based Integrated Satellite Terrestrial Network |
| title_short | Constrained DRL for Energy Efficiency Optimization in RSMA-Based Integrated Satellite Terrestrial Network |
| title_sort | constrained drl for energy efficiency optimization in rsma based integrated satellite terrestrial network |
| topic | integrated satellite terrestrial network rate splitting multiple access energy efficiency constrained deep reinforcement learning soft actor-critic |
| url | https://www.mdpi.com/1424-8220/23/18/7859 |
| work_keys_str_mv | AT qingmiaozhang constraineddrlforenergyefficiencyoptimizationinrsmabasedintegratedsatelliteterrestrialnetwork AT lidongzhu constraineddrlforenergyefficiencyoptimizationinrsmabasedintegratedsatelliteterrestrialnetwork AT yanyanchen constraineddrlforenergyefficiencyoptimizationinrsmabasedintegratedsatelliteterrestrialnetwork AT shanjiang constraineddrlforenergyefficiencyoptimizationinrsmabasedintegratedsatelliteterrestrialnetwork |