Dynamic deployment method based on double deep Q-network in UAV-assisted MEC systems
Abstract The unmanned aerial vehicle (UAV) assisted mobile edge computing (MEC) system leverages the high maneuverability of UAVs to provide efficient computing services to terminals. A dynamic deployment algorithm based on double deep Q-networks (DDQN) is suggested to address issues with energy lim...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2023-09-01
|
| Series: | Journal of Cloud Computing: Advances, Systems and Applications |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13677-023-00507-6 |
| _version_ | 1797556473647071232 |
|---|---|
| author | Suqin Zhang Lin Zhang Fei Xu Song Cheng Weiya Su Sen Wang |
| author_facet | Suqin Zhang Lin Zhang Fei Xu Song Cheng Weiya Su Sen Wang |
| author_sort | Suqin Zhang |
| collection | DOAJ |
| description | Abstract The unmanned aerial vehicle (UAV) assisted mobile edge computing (MEC) system leverages the high maneuverability of UAVs to provide efficient computing services to terminals. A dynamic deployment algorithm based on double deep Q-networks (DDQN) is suggested to address issues with energy limitation and obstacle avoidance when providing edge services to terminals by UAV. First, the energy consumption of the UAV and the fairness of the terminal’s geographic location are jointly optimized in the case of multiple obstacles and multiple terminals on the ground. And the UAV can avoid obstacles. Furthermore, a double deep Q-network was introduced to address the slow convergence and risk of falling into local optima during the optimization problem training process. Also included in the learning process was a pseudo count exploration strategy. Finally, the improved DDQN algorithm achieves faster convergence and a higher average system reward, according to experimental results. Regarding the fairness of geographic locations of terminals, the improved DDQN algorithm outperforms Q-learning, DQN, and DDQN algorithms by 50%, 20%, and 15.38%, respectively, and the stability of the improved algorithm is also validated. |
| first_indexed | 2024-03-10T17:02:22Z |
| format | Article |
| id | doaj.art-e85c3a3d3e46411bb2e203fee5107226 |
| institution | Directory Open Access Journal |
| issn | 2192-113X |
| language | English |
| last_indexed | 2024-03-10T17:02:22Z |
| publishDate | 2023-09-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Journal of Cloud Computing: Advances, Systems and Applications |
| spelling | doaj.art-e85c3a3d3e46411bb2e203fee51072262023-11-20T10:56:00ZengSpringerOpenJournal of Cloud Computing: Advances, Systems and Applications2192-113X2023-09-0112111610.1186/s13677-023-00507-6Dynamic deployment method based on double deep Q-network in UAV-assisted MEC systemsSuqin Zhang0Lin Zhang1Fei Xu2Song Cheng3Weiya Su4Sen Wang5School of Basic, Xi’an Technological UniversitySchool of Ordnance Science and Technology, Xi’an Technological UniversitySchool of Computer Science and Engineering, Xi’an Technological UniversitySchool of Ordnance Science and Technology, Xi’an Technological UniversitySchool of Computer Science and Engineering, Xi’an Technological UniversitySchool of Ordnance Science and Technology, Xi’an Technological UniversityAbstract The unmanned aerial vehicle (UAV) assisted mobile edge computing (MEC) system leverages the high maneuverability of UAVs to provide efficient computing services to terminals. A dynamic deployment algorithm based on double deep Q-networks (DDQN) is suggested to address issues with energy limitation and obstacle avoidance when providing edge services to terminals by UAV. First, the energy consumption of the UAV and the fairness of the terminal’s geographic location are jointly optimized in the case of multiple obstacles and multiple terminals on the ground. And the UAV can avoid obstacles. Furthermore, a double deep Q-network was introduced to address the slow convergence and risk of falling into local optima during the optimization problem training process. Also included in the learning process was a pseudo count exploration strategy. Finally, the improved DDQN algorithm achieves faster convergence and a higher average system reward, according to experimental results. Regarding the fairness of geographic locations of terminals, the improved DDQN algorithm outperforms Q-learning, DQN, and DDQN algorithms by 50%, 20%, and 15.38%, respectively, and the stability of the improved algorithm is also validated.https://doi.org/10.1186/s13677-023-00507-6Dynamic deploymentUnmanned aerial vehicle (UAV)Mobile edge computing (MEC)Double deep Q-network |
| spellingShingle | Suqin Zhang Lin Zhang Fei Xu Song Cheng Weiya Su Sen Wang Dynamic deployment method based on double deep Q-network in UAV-assisted MEC systems Journal of Cloud Computing: Advances, Systems and Applications Dynamic deployment Unmanned aerial vehicle (UAV) Mobile edge computing (MEC) Double deep Q-network |
| title | Dynamic deployment method based on double deep Q-network in UAV-assisted MEC systems |
| title_full | Dynamic deployment method based on double deep Q-network in UAV-assisted MEC systems |
| title_fullStr | Dynamic deployment method based on double deep Q-network in UAV-assisted MEC systems |
| title_full_unstemmed | Dynamic deployment method based on double deep Q-network in UAV-assisted MEC systems |
| title_short | Dynamic deployment method based on double deep Q-network in UAV-assisted MEC systems |
| title_sort | dynamic deployment method based on double deep q network in uav assisted mec systems |
| topic | Dynamic deployment Unmanned aerial vehicle (UAV) Mobile edge computing (MEC) Double deep Q-network |
| url | https://doi.org/10.1186/s13677-023-00507-6 |
| work_keys_str_mv | AT suqinzhang dynamicdeploymentmethodbasedondoubledeepqnetworkinuavassistedmecsystems AT linzhang dynamicdeploymentmethodbasedondoubledeepqnetworkinuavassistedmecsystems AT feixu dynamicdeploymentmethodbasedondoubledeepqnetworkinuavassistedmecsystems AT songcheng dynamicdeploymentmethodbasedondoubledeepqnetworkinuavassistedmecsystems AT weiyasu dynamicdeploymentmethodbasedondoubledeepqnetworkinuavassistedmecsystems AT senwang dynamicdeploymentmethodbasedondoubledeepqnetworkinuavassistedmecsystems |