Computation Offloading and Resource Allocation Based on P-DQN in LEO Satellite Edge Networks
Traditional low earth orbit (LEO) satellite networks are typically independent of terrestrial networks, which develop relatively slowly due to the on-board capacity limitation. By integrating emerging mobile edge computing (MEC) with LEO satellite networks to form the business-oriented “end-edge-clo...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/24/9885 |
| _version_ | 1827573535792431104 |
|---|---|
| author | Xu Yang Hai Fang Yuan Gao Xingjie Wang Kan Wang Zheng Liu |
| author_facet | Xu Yang Hai Fang Yuan Gao Xingjie Wang Kan Wang Zheng Liu |
| author_sort | Xu Yang |
| collection | DOAJ |
| description | Traditional low earth orbit (LEO) satellite networks are typically independent of terrestrial networks, which develop relatively slowly due to the on-board capacity limitation. By integrating emerging mobile edge computing (MEC) with LEO satellite networks to form the business-oriented “end-edge-cloud” multi-level computing architecture, some computing-sensitive tasks can be offloaded by ground terminals to satellites, thereby satisfying more tasks in the network. How to make computation offloading and resource allocation decisions in LEO satellite edge networks, nevertheless, indeed poses challenges in tracking network dynamics and handling sophisticated actions. For the discrete-continuous hybrid action space and time-varying networks, this work aims to use the parameterized deep Q-network (P-DQN) for the joint computation offloading and resource allocation. First, the characteristics of time-varying channels are modeled, and then both communication and computation models under three different offloading decisions are constructed. Second, the constraints on task offloading decisions, on remaining available computing resources, and on the power control of LEO satellites as well as the cloud server are formulated, followed by the maximization problem of satisfied task number over the long run. Third, using the parameterized action Markov decision process (PAMDP) and P-DQN, the joint computing offloading, resource allocation, and power control are made in real time, to accommodate dynamics in LEO satellite edge networks and dispose of the discrete-continuous hybrid action space. Simulation results show that the proposed P-DQN method could approach the optimal control, and outperforms other reinforcement learning (RL) methods for merely either discrete or continuous action space, in terms of the long-term rate of satisfied tasks. |
| first_indexed | 2024-03-08T20:21:37Z |
| format | Article |
| id | doaj.art-02edafbd84df407cb7422d156605b1ac |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-08T20:21:37Z |
| publishDate | 2023-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-02edafbd84df407cb7422d156605b1ac2023-12-22T14:41:29ZengMDPI AGSensors1424-82202023-12-012324988510.3390/s23249885Computation Offloading and Resource Allocation Based on P-DQN in LEO Satellite Edge NetworksXu Yang0Hai Fang1Yuan Gao2Xingjie Wang3Kan Wang4Zheng Liu5Xi’an Institute of Space Radio Technology, Xi’an 710100, ChinaXi’an Institute of Space Radio Technology, Xi’an 710100, ChinaXi’an Institute of Space Radio Technology, Xi’an 710100, ChinaSchool of Computer Science and Engineering, Xi’an University of Technology, Xi’an 710048, ChinaSchool of Computer Science and Engineering, Xi’an University of Technology, Xi’an 710048, ChinaSchool of Computer Science and Engineering, Xi’an University of Technology, Xi’an 710048, ChinaTraditional low earth orbit (LEO) satellite networks are typically independent of terrestrial networks, which develop relatively slowly due to the on-board capacity limitation. By integrating emerging mobile edge computing (MEC) with LEO satellite networks to form the business-oriented “end-edge-cloud” multi-level computing architecture, some computing-sensitive tasks can be offloaded by ground terminals to satellites, thereby satisfying more tasks in the network. How to make computation offloading and resource allocation decisions in LEO satellite edge networks, nevertheless, indeed poses challenges in tracking network dynamics and handling sophisticated actions. For the discrete-continuous hybrid action space and time-varying networks, this work aims to use the parameterized deep Q-network (P-DQN) for the joint computation offloading and resource allocation. First, the characteristics of time-varying channels are modeled, and then both communication and computation models under three different offloading decisions are constructed. Second, the constraints on task offloading decisions, on remaining available computing resources, and on the power control of LEO satellites as well as the cloud server are formulated, followed by the maximization problem of satisfied task number over the long run. Third, using the parameterized action Markov decision process (PAMDP) and P-DQN, the joint computing offloading, resource allocation, and power control are made in real time, to accommodate dynamics in LEO satellite edge networks and dispose of the discrete-continuous hybrid action space. Simulation results show that the proposed P-DQN method could approach the optimal control, and outperforms other reinforcement learning (RL) methods for merely either discrete or continuous action space, in terms of the long-term rate of satisfied tasks.https://www.mdpi.com/1424-8220/23/24/9885LEO satellite edge networksoffloading decisionresource allocationhybrid action spaceP-DQN |
| spellingShingle | Xu Yang Hai Fang Yuan Gao Xingjie Wang Kan Wang Zheng Liu Computation Offloading and Resource Allocation Based on P-DQN in LEO Satellite Edge Networks Sensors LEO satellite edge networks offloading decision resource allocation hybrid action space P-DQN |
| title | Computation Offloading and Resource Allocation Based on P-DQN in LEO Satellite Edge Networks |
| title_full | Computation Offloading and Resource Allocation Based on P-DQN in LEO Satellite Edge Networks |
| title_fullStr | Computation Offloading and Resource Allocation Based on P-DQN in LEO Satellite Edge Networks |
| title_full_unstemmed | Computation Offloading and Resource Allocation Based on P-DQN in LEO Satellite Edge Networks |
| title_short | Computation Offloading and Resource Allocation Based on P-DQN in LEO Satellite Edge Networks |
| title_sort | computation offloading and resource allocation based on p dqn in leo satellite edge networks |
| topic | LEO satellite edge networks offloading decision resource allocation hybrid action space P-DQN |
| url | https://www.mdpi.com/1424-8220/23/24/9885 |
| work_keys_str_mv | AT xuyang computationoffloadingandresourceallocationbasedonpdqninleosatelliteedgenetworks AT haifang computationoffloadingandresourceallocationbasedonpdqninleosatelliteedgenetworks AT yuangao computationoffloadingandresourceallocationbasedonpdqninleosatelliteedgenetworks AT xingjiewang computationoffloadingandresourceallocationbasedonpdqninleosatelliteedgenetworks AT kanwang computationoffloadingandresourceallocationbasedonpdqninleosatelliteedgenetworks AT zhengliu computationoffloadingandresourceallocationbasedonpdqninleosatelliteedgenetworks |