Deep-Deterministic Policy Gradient Based Multi-Resource Allocation in Edge-Cloud System: A Distributed Approach
Edge Cloud (EC) empowers the beyond 5G (B5G) wireless networks to cope with large-scale and real-time traffics of Internet-of-Things (IoT) by minimizing the latency and providing compute power at the edge of the network. Due to a limited amount of resources at the EC compared to the back-end cloud (...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10054044/ |
| _version_ | 1811160973010010112 |
|---|---|
| author | Arslan Qadeer Myung Jong Lee |
| author_facet | Arslan Qadeer Myung Jong Lee |
| author_sort | Arslan Qadeer |
| collection | DOAJ |
| description | Edge Cloud (EC) empowers the beyond 5G (B5G) wireless networks to cope with large-scale and real-time traffics of Internet-of-Things (IoT) by minimizing the latency and providing compute power at the edge of the network. Due to a limited amount of resources at the EC compared to the back-end cloud (BC), intelligent resource management techniques become imperative. This paper studies the problem of multi-resource allocation (MRA) in terms of compute and wireless resources in an integrated EC and BC environment. Machine learning-based approaches are emerging to solve such optimization problems. However, it is challenging to adopt traditional discrete action space-based methods due to their high dimensionality issue. To this end, we propose a deep-deterministic policy gradient (DDPG) based temporal feature learning attentional network (TFLAN) model to address the MRA problem. TFLAN combines convolution, gated recurrent unit and attention layers together to mine local and long term temporal information from the task sequences for excellent function approximation. A novel heuristic-based priority experience replay (hPER) method is formulated to accelerate the convergence speed. Further, a pruning principle helps the TFLAN agent to significantly reduce the computational complexity and balance the load among base stations and servers to minimize the rejection-rate. Lastly, data parallelism technique is adopted for distributed training to meet the needs of a high-volume of IoT traffic in the EC environment. Experimental results demonstrate that the distributed training approach suites well to the problem scale and can magnify the speed of the learning process. We validate the proposed framework by comparing with five state-of-the-art RL agents. Our proposed agent converges fast and achieves up to 28% and 72% reduction in operational cost and rejection-rate, and achieves up to 32% gain in the quality of experience on average, compared to the most advanced DDPG agent. |
| first_indexed | 2024-04-10T06:06:47Z |
| format | Article |
| id | doaj.art-efce5bf878814ab98a3160faea37b050 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-10T06:06:47Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-efce5bf878814ab98a3160faea37b0502023-03-03T00:00:19ZengIEEEIEEE Access2169-35362023-01-0111203812039810.1109/ACCESS.2023.324915310054044Deep-Deterministic Policy Gradient Based Multi-Resource Allocation in Edge-Cloud System: A Distributed ApproachArslan Qadeer0https://orcid.org/0000-0003-2399-3618Myung Jong Lee1https://orcid.org/0000-0001-6045-7764Department of Electrical Engineering, The City College of New York, New York, NY, USADepartment of Electrical Engineering, The City College of New York, New York, NY, USAEdge Cloud (EC) empowers the beyond 5G (B5G) wireless networks to cope with large-scale and real-time traffics of Internet-of-Things (IoT) by minimizing the latency and providing compute power at the edge of the network. Due to a limited amount of resources at the EC compared to the back-end cloud (BC), intelligent resource management techniques become imperative. This paper studies the problem of multi-resource allocation (MRA) in terms of compute and wireless resources in an integrated EC and BC environment. Machine learning-based approaches are emerging to solve such optimization problems. However, it is challenging to adopt traditional discrete action space-based methods due to their high dimensionality issue. To this end, we propose a deep-deterministic policy gradient (DDPG) based temporal feature learning attentional network (TFLAN) model to address the MRA problem. TFLAN combines convolution, gated recurrent unit and attention layers together to mine local and long term temporal information from the task sequences for excellent function approximation. A novel heuristic-based priority experience replay (hPER) method is formulated to accelerate the convergence speed. Further, a pruning principle helps the TFLAN agent to significantly reduce the computational complexity and balance the load among base stations and servers to minimize the rejection-rate. Lastly, data parallelism technique is adopted for distributed training to meet the needs of a high-volume of IoT traffic in the EC environment. Experimental results demonstrate that the distributed training approach suites well to the problem scale and can magnify the speed of the learning process. We validate the proposed framework by comparing with five state-of-the-art RL agents. Our proposed agent converges fast and achieves up to 28% and 72% reduction in operational cost and rejection-rate, and achieves up to 32% gain in the quality of experience on average, compared to the most advanced DDPG agent.https://ieeexplore.ieee.org/document/10054044/Edge cloud computingwireless networksdeep deterministic policy gradientresource allocationsmart cityIoT |
| spellingShingle | Arslan Qadeer Myung Jong Lee Deep-Deterministic Policy Gradient Based Multi-Resource Allocation in Edge-Cloud System: A Distributed Approach IEEE Access Edge cloud computing wireless networks deep deterministic policy gradient resource allocation smart city IoT |
| title | Deep-Deterministic Policy Gradient Based Multi-Resource Allocation in Edge-Cloud System: A Distributed Approach |
| title_full | Deep-Deterministic Policy Gradient Based Multi-Resource Allocation in Edge-Cloud System: A Distributed Approach |
| title_fullStr | Deep-Deterministic Policy Gradient Based Multi-Resource Allocation in Edge-Cloud System: A Distributed Approach |
| title_full_unstemmed | Deep-Deterministic Policy Gradient Based Multi-Resource Allocation in Edge-Cloud System: A Distributed Approach |
| title_short | Deep-Deterministic Policy Gradient Based Multi-Resource Allocation in Edge-Cloud System: A Distributed Approach |
| title_sort | deep deterministic policy gradient based multi resource allocation in edge cloud system a distributed approach |
| topic | Edge cloud computing wireless networks deep deterministic policy gradient resource allocation smart city IoT |
| url | https://ieeexplore.ieee.org/document/10054044/ |
| work_keys_str_mv | AT arslanqadeer deepdeterministicpolicygradientbasedmultiresourceallocationinedgecloudsystemadistributedapproach AT myungjonglee deepdeterministicpolicygradientbasedmultiresourceallocationinedgecloudsystemadistributedapproach |