A DRL Strategy for Optimal Resource Allocation Along With 3D Trajectory Dynamics in UAV-MEC Network
Advances in Unmanned Air Vehicle (UAV) technology have paved a way for numerous configurations and applications in communication systems. However, UAV dynamics play an important role in determining its effective use. In this article, while considering UAV dynamics, we evaluate the performance of a U...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10129264/ |
| _version_ | 1827930597886001152 |
|---|---|
| author | Tayyaba Khurshid Waqas Ahmed Muhammad Rehan Rizwan Ahmad Muhammad Mahtab Alam Ayman Radwan |
| author_facet | Tayyaba Khurshid Waqas Ahmed Muhammad Rehan Rizwan Ahmad Muhammad Mahtab Alam Ayman Radwan |
| author_sort | Tayyaba Khurshid |
| collection | DOAJ |
| description | Advances in Unmanned Air Vehicle (UAV) technology have paved a way for numerous configurations and applications in communication systems. However, UAV dynamics play an important role in determining its effective use. In this article, while considering UAV dynamics, we evaluate the performance of a UAV equipped with a Mobile-Edge Computing (MEC) server that provides services to End-user Devices (EuDs). The EuDs due to their limited energy resources offload a portion of their computational task to nearby MEC-based UAV. To this end, we jointly optimize the computational cost and 3D UAV placement along with resource allocation subject to the network, communication, and environment constraints. A Deep Reinforcement Learning (DRL) technique based on a continuous action space approach, namely Deep Deterministic Policy Gradient (DDPG) is utilized. By exploiting DDPG, we propose an optimization strategy to obtain an optimal offloading policy in the presence of UAV dynamics, which is not considered in earlier studies. The proposed strategy can be classified into three cases namely; training through an ideal scenario, training through error dynamics, and training through extreme values. We compared the performance of these individual cases based on cost percentage and concluded that case II (training through error dynamics) achieves minimum cost i.e., 37.75 %, whereas case I and case III settles at 67.25% and 67.50% respectively. Numerical simulations are performed, and extensive results are obtained which shows that the advanced DDPG based algorithm along with error dynamic protocol is able to converge to near optimum. To validate the efficacy of the proposed algorithm, a comparison with state-of-the-art Deep Q-Network (DQN) is carried out, which shows that our algorithm has significant improvements. |
| first_indexed | 2024-03-13T06:37:58Z |
| format | Article |
| id | doaj.art-4eadd9e1b3ed44edb4f312b48da85154 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-13T06:37:58Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-4eadd9e1b3ed44edb4f312b48da851542023-06-08T23:00:39ZengIEEEIEEE Access2169-35362023-01-0111546645467810.1109/ACCESS.2023.327859110129264A DRL Strategy for Optimal Resource Allocation Along With 3D Trajectory Dynamics in UAV-MEC NetworkTayyaba Khurshid0Waqas Ahmed1https://orcid.org/0000-0001-5928-4245Muhammad Rehan2https://orcid.org/0000-0002-9908-3971Rizwan Ahmad3https://orcid.org/0000-0002-4758-7895Muhammad Mahtab Alam4https://orcid.org/0000-0002-1055-7959Ayman Radwan5https://orcid.org/0000-0003-1935-6077Department of Electrical Engineering (DEE), Pakistan Institute of Engineering and Applied Sciences (PIEAS), Islamabad, PakistanDepartment of Electrical Engineering (DEE), Pakistan Institute of Engineering and Applied Sciences (PIEAS), Islamabad, PakistanDepartment of Electrical Engineering (DEE), Pakistan Institute of Engineering and Applied Sciences (PIEAS), Islamabad, PakistanSchool of Electrical Engineering and Computer Science, National University of Sciences and Technology (NUST), Islamabad, PakistanThomas Johann Seebeck Department of Electronics, Tallinn University of Technology, Tallinn, EstoniaDepartment of Electronics, Telecommunications, and Informatics (DETI), University of Aveiro, and Instituto de Telecomunicações, Aveiro, PortugalAdvances in Unmanned Air Vehicle (UAV) technology have paved a way for numerous configurations and applications in communication systems. However, UAV dynamics play an important role in determining its effective use. In this article, while considering UAV dynamics, we evaluate the performance of a UAV equipped with a Mobile-Edge Computing (MEC) server that provides services to End-user Devices (EuDs). The EuDs due to their limited energy resources offload a portion of their computational task to nearby MEC-based UAV. To this end, we jointly optimize the computational cost and 3D UAV placement along with resource allocation subject to the network, communication, and environment constraints. A Deep Reinforcement Learning (DRL) technique based on a continuous action space approach, namely Deep Deterministic Policy Gradient (DDPG) is utilized. By exploiting DDPG, we propose an optimization strategy to obtain an optimal offloading policy in the presence of UAV dynamics, which is not considered in earlier studies. The proposed strategy can be classified into three cases namely; training through an ideal scenario, training through error dynamics, and training through extreme values. We compared the performance of these individual cases based on cost percentage and concluded that case II (training through error dynamics) achieves minimum cost i.e., 37.75 %, whereas case I and case III settles at 67.25% and 67.50% respectively. Numerical simulations are performed, and extensive results are obtained which shows that the advanced DDPG based algorithm along with error dynamic protocol is able to converge to near optimum. To validate the efficacy of the proposed algorithm, a comparison with state-of-the-art Deep Q-Network (DQN) is carried out, which shows that our algorithm has significant improvements.https://ieeexplore.ieee.org/document/10129264/MECoffloading ratioresource allocationtrajectory optimizationUAV dynamics |
| spellingShingle | Tayyaba Khurshid Waqas Ahmed Muhammad Rehan Rizwan Ahmad Muhammad Mahtab Alam Ayman Radwan A DRL Strategy for Optimal Resource Allocation Along With 3D Trajectory Dynamics in UAV-MEC Network IEEE Access MEC offloading ratio resource allocation trajectory optimization UAV dynamics |
| title | A DRL Strategy for Optimal Resource Allocation Along With 3D Trajectory Dynamics in UAV-MEC Network |
| title_full | A DRL Strategy for Optimal Resource Allocation Along With 3D Trajectory Dynamics in UAV-MEC Network |
| title_fullStr | A DRL Strategy for Optimal Resource Allocation Along With 3D Trajectory Dynamics in UAV-MEC Network |
| title_full_unstemmed | A DRL Strategy for Optimal Resource Allocation Along With 3D Trajectory Dynamics in UAV-MEC Network |
| title_short | A DRL Strategy for Optimal Resource Allocation Along With 3D Trajectory Dynamics in UAV-MEC Network |
| title_sort | drl strategy for optimal resource allocation along with 3d trajectory dynamics in uav mec network |
| topic | MEC offloading ratio resource allocation trajectory optimization UAV dynamics |
| url | https://ieeexplore.ieee.org/document/10129264/ |
| work_keys_str_mv | AT tayyabakhurshid adrlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT waqasahmed adrlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT muhammadrehan adrlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT rizwanahmad adrlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT muhammadmahtabalam adrlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT aymanradwan adrlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT tayyabakhurshid drlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT waqasahmed drlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT muhammadrehan drlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT rizwanahmad drlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT muhammadmahtabalam drlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork AT aymanradwan drlstrategyforoptimalresourceallocationalongwith3dtrajectorydynamicsinuavmecnetwork |