DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET
Unmanned aerial vehicles (UAVs) are important equipment for efficiently executing search and rescue missions in disaster or air-crash scenarios. Each node can communicate with the others by a routing protocol in UAV ad hoc networks (UANETs). However, UAV routing protocols are faced with the challeng...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/13/5020 |
| _version_ | 1797434054686015488 |
|---|---|
| author | Yanan Zhang Hongbing Qiu |
| author_facet | Yanan Zhang Hongbing Qiu |
| author_sort | Yanan Zhang |
| collection | DOAJ |
| description | Unmanned aerial vehicles (UAVs) are important equipment for efficiently executing search and rescue missions in disaster or air-crash scenarios. Each node can communicate with the others by a routing protocol in UAV ad hoc networks (UANETs). However, UAV routing protocols are faced with the challenges of high mobility and limited node energy, which hugely lead to unstable link and sparse network topology due to premature node death. Eventually, this severely affects network performance. In order to solve these problems, we proposed the deep-reinforcement-learning-based geographical routing protocol of considering link stability and energy prediction (DSEGR) for UANETs. First of all, we came up with the link stability evaluation indicator and utilized the autoregressive integrated moving average (ARIMA) model to predict the residual energy of neighbor nodes. Then, the packet forward process was modeled as a Markov Decision Process, and according to a deep double Q network with prioritized experience replay to learn the routing-decision process. Meanwhile, a reward function was designed to obtain a better convergence rate, and the analytic hierarchy process (AHP) was used to analyze the weights of the considered factors in the reward function. Finally, to verify the effectiveness of DSEGR, we conducted simulation experiments to analyze network performance. The simulation results demonstrate that our proposed routing protocol remarkably outperforms others in packet delivery ratio and has a faster convergence rate. |
| first_indexed | 2024-03-09T10:25:48Z |
| format | Article |
| id | doaj.art-f7cccc8827dc46c8ae4c5fc48801b289 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T10:25:48Z |
| publishDate | 2022-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-f7cccc8827dc46c8ae4c5fc48801b2892023-12-01T21:42:14ZengMDPI AGSensors1424-82202022-07-012213502010.3390/s22135020DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANETYanan Zhang0Hongbing Qiu1School of Information and Communications, Guilin University of Electronic Technology, Guilin 541004, ChinaSchool of Information and Communications, Guilin University of Electronic Technology, Guilin 541004, ChinaUnmanned aerial vehicles (UAVs) are important equipment for efficiently executing search and rescue missions in disaster or air-crash scenarios. Each node can communicate with the others by a routing protocol in UAV ad hoc networks (UANETs). However, UAV routing protocols are faced with the challenges of high mobility and limited node energy, which hugely lead to unstable link and sparse network topology due to premature node death. Eventually, this severely affects network performance. In order to solve these problems, we proposed the deep-reinforcement-learning-based geographical routing protocol of considering link stability and energy prediction (DSEGR) for UANETs. First of all, we came up with the link stability evaluation indicator and utilized the autoregressive integrated moving average (ARIMA) model to predict the residual energy of neighbor nodes. Then, the packet forward process was modeled as a Markov Decision Process, and according to a deep double Q network with prioritized experience replay to learn the routing-decision process. Meanwhile, a reward function was designed to obtain a better convergence rate, and the analytic hierarchy process (AHP) was used to analyze the weights of the considered factors in the reward function. Finally, to verify the effectiveness of DSEGR, we conducted simulation experiments to analyze network performance. The simulation results demonstrate that our proposed routing protocol remarkably outperforms others in packet delivery ratio and has a faster convergence rate.https://www.mdpi.com/1424-8220/22/13/5020unmanned aerial vehicles (UAVs)routing protocoldeep double Q network (DDQN)prioritized experience replaylink stabilityautoregressive integrated moving average (ARIMA) |
| spellingShingle | Yanan Zhang Hongbing Qiu DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET Sensors unmanned aerial vehicles (UAVs) routing protocol deep double Q network (DDQN) prioritized experience replay link stability autoregressive integrated moving average (ARIMA) |
| title | DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET |
| title_full | DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET |
| title_fullStr | DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET |
| title_full_unstemmed | DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET |
| title_short | DDQN with Prioritized Experience Replay-Based Optimized Geographical Routing Protocol of Considering Link Stability and Energy Prediction for UANET |
| title_sort | ddqn with prioritized experience replay based optimized geographical routing protocol of considering link stability and energy prediction for uanet |
| topic | unmanned aerial vehicles (UAVs) routing protocol deep double Q network (DDQN) prioritized experience replay link stability autoregressive integrated moving average (ARIMA) |
| url | https://www.mdpi.com/1424-8220/22/13/5020 |
| work_keys_str_mv | AT yananzhang ddqnwithprioritizedexperiencereplaybasedoptimizedgeographicalroutingprotocolofconsideringlinkstabilityandenergypredictionforuanet AT hongbingqiu ddqnwithprioritizedexperiencereplaybasedoptimizedgeographicalroutingprotocolofconsideringlinkstabilityandenergypredictionforuanet |