A novel Q-learning-based routing scheme using an intelligent filtering algorithm for flying ad hoc networks (FANETs)
The flying ad hoc network (FANET) is an emerging network focused on unmanned aerial vehicles (UAVs) that has attracted the attention of researchers around the world. Due to the cooperation between UAVs in this network, data transfer between these UAVs is very essential. Routing protocols must determ...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-12-01
|
| Series: | Journal of King Saud University: Computer and Information Sciences |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1319157823003713 |
| _version_ | 1797389544545320960 |
|---|---|
| author | Mehdi Hosseinzadeh Saqib Ali Liliana Ionescu-Feleaga Bogdan-Stefan Ionescu Mohammad Sadegh Yousefpoor Efat Yousefpoor Omed Hassan Ahmed Amir Masoud Rahmani Asif Mehmood |
| author_facet | Mehdi Hosseinzadeh Saqib Ali Liliana Ionescu-Feleaga Bogdan-Stefan Ionescu Mohammad Sadegh Yousefpoor Efat Yousefpoor Omed Hassan Ahmed Amir Masoud Rahmani Asif Mehmood |
| author_sort | Mehdi Hosseinzadeh |
| collection | DOAJ |
| description | The flying ad hoc network (FANET) is an emerging network focused on unmanned aerial vehicles (UAVs) that has attracted the attention of researchers around the world. Due to the cooperation between UAVs in this network, data transfer between these UAVs is very essential. Routing protocols must determine how to make routing paths for each UAV with others in a wireless ad hoc network to facilitate the data transmission between UAVs. Nowadays, reinforcement learning (RL), especially Q-learning, is an effective response for solving existing challenges in the routing approaches and adding features such as autonomous, self-adaptive, and self-learning to these approaches. In this paper, Q-learning is used to enhance and increase network performance, and a Q-learning-based routing method using an intelligent filtering algorithm called QRF is presented for FANETs. The main innovation in this paper is that QRF manages the size of the state space using the proposed filtering algorithm. This will increase the convergence rate of the Q-learning-based routing algorithm. On the other hand, QRF regulates the learning parameters related to Q-learning so that this scheme is better adapted to the FANET environment. In the last step, the network simulator version 2 (NS2) is employed to execute the simulation process related to QRF. In this process, five evaluation criteria, namely energy consumption, packet delivery rate, overhead, end-to-end delay, and network longevity are evaluated, and the results obtained from QRF are compared with those of QFAN, QTAR, and QGeo. The simulation results in this paper show that QRF makes a balanced energy distribution between UAVs and thus extends the network longevity. Moreover, the intelligent filtering algorithm designed in QRF has reduced delay in the routing process but is associated with communication overhead. |
| first_indexed | 2024-03-08T22:57:27Z |
| format | Article |
| id | doaj.art-59b76505fa234c8399ed1f16c8d51a21 |
| institution | Directory Open Access Journal |
| issn | 1319-1578 |
| language | English |
| last_indexed | 2024-03-08T22:57:27Z |
| publishDate | 2023-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Journal of King Saud University: Computer and Information Sciences |
| spelling | doaj.art-59b76505fa234c8399ed1f16c8d51a212023-12-16T06:06:05ZengElsevierJournal of King Saud University: Computer and Information Sciences1319-15782023-12-013510101817A novel Q-learning-based routing scheme using an intelligent filtering algorithm for flying ad hoc networks (FANETs)Mehdi Hosseinzadeh0Saqib Ali1Liliana Ionescu-Feleaga2Bogdan-Stefan Ionescu3Mohammad Sadegh Yousefpoor4Efat Yousefpoor5Omed Hassan Ahmed6Amir Masoud Rahmani7Asif Mehmood8Institute of Research and Development, Duy Tan University, Da Nang, Viet Nam; School of Medicine and Pharmacy, Duy Tan University, Da Nang, Viet NamDepartment of Information Systems, College of Economics and Political Science, Sultan Qaboos University, Al Khoudh, Muscat, OmanDepartment of Accounting and Audit, Bucharest University of Economic Studies, 010374 Bucharest, RomaniaDepartment of Management Information System, Bucharest University of Economic Studies, 010374 Bucharest, RomaniaDepartment of Computer Engineering, Dezful Branch, Islamic Azad University, Dezful, IranDepartment of Computer Engineering, Dezful Branch, Islamic Azad University, Dezful, IranDepartment of Information Technology, University of Human Development Sulaymaniyah, IraqFuture Technology Research Center, National Yunlin University of Science and Technology, Yunlin, Taiwan; Corresponding authors.Department of Biomedical Engineering, Gachon University, Seongnam-si, Gyeonggi-do 13120, Republic of Korea; Corresponding authors.The flying ad hoc network (FANET) is an emerging network focused on unmanned aerial vehicles (UAVs) that has attracted the attention of researchers around the world. Due to the cooperation between UAVs in this network, data transfer between these UAVs is very essential. Routing protocols must determine how to make routing paths for each UAV with others in a wireless ad hoc network to facilitate the data transmission between UAVs. Nowadays, reinforcement learning (RL), especially Q-learning, is an effective response for solving existing challenges in the routing approaches and adding features such as autonomous, self-adaptive, and self-learning to these approaches. In this paper, Q-learning is used to enhance and increase network performance, and a Q-learning-based routing method using an intelligent filtering algorithm called QRF is presented for FANETs. The main innovation in this paper is that QRF manages the size of the state space using the proposed filtering algorithm. This will increase the convergence rate of the Q-learning-based routing algorithm. On the other hand, QRF regulates the learning parameters related to Q-learning so that this scheme is better adapted to the FANET environment. In the last step, the network simulator version 2 (NS2) is employed to execute the simulation process related to QRF. In this process, five evaluation criteria, namely energy consumption, packet delivery rate, overhead, end-to-end delay, and network longevity are evaluated, and the results obtained from QRF are compared with those of QFAN, QTAR, and QGeo. The simulation results in this paper show that QRF makes a balanced energy distribution between UAVs and thus extends the network longevity. Moreover, the intelligent filtering algorithm designed in QRF has reduced delay in the routing process but is associated with communication overhead.http://www.sciencedirect.com/science/article/pii/S1319157823003713Flying ad hoc networks (FANETs)Reinforcement learning (RL)Q-learningRoutingUnmanned aerial vehicles (UAVs) |
| spellingShingle | Mehdi Hosseinzadeh Saqib Ali Liliana Ionescu-Feleaga Bogdan-Stefan Ionescu Mohammad Sadegh Yousefpoor Efat Yousefpoor Omed Hassan Ahmed Amir Masoud Rahmani Asif Mehmood A novel Q-learning-based routing scheme using an intelligent filtering algorithm for flying ad hoc networks (FANETs) Journal of King Saud University: Computer and Information Sciences Flying ad hoc networks (FANETs) Reinforcement learning (RL) Q-learning Routing Unmanned aerial vehicles (UAVs) |
| title | A novel Q-learning-based routing scheme using an intelligent filtering algorithm for flying ad hoc networks (FANETs) |
| title_full | A novel Q-learning-based routing scheme using an intelligent filtering algorithm for flying ad hoc networks (FANETs) |
| title_fullStr | A novel Q-learning-based routing scheme using an intelligent filtering algorithm for flying ad hoc networks (FANETs) |
| title_full_unstemmed | A novel Q-learning-based routing scheme using an intelligent filtering algorithm for flying ad hoc networks (FANETs) |
| title_short | A novel Q-learning-based routing scheme using an intelligent filtering algorithm for flying ad hoc networks (FANETs) |
| title_sort | novel q learning based routing scheme using an intelligent filtering algorithm for flying ad hoc networks fanets |
| topic | Flying ad hoc networks (FANETs) Reinforcement learning (RL) Q-learning Routing Unmanned aerial vehicles (UAVs) |
| url | http://www.sciencedirect.com/science/article/pii/S1319157823003713 |
| work_keys_str_mv | AT mehdihosseinzadeh anovelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT saqibali anovelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT lilianaionescufeleaga anovelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT bogdanstefanionescu anovelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT mohammadsadeghyousefpoor anovelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT efatyousefpoor anovelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT omedhassanahmed anovelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT amirmasoudrahmani anovelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT asifmehmood anovelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT mehdihosseinzadeh novelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT saqibali novelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT lilianaionescufeleaga novelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT bogdanstefanionescu novelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT mohammadsadeghyousefpoor novelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT efatyousefpoor novelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT omedhassanahmed novelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT amirmasoudrahmani novelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets AT asifmehmood novelqlearningbasedroutingschemeusinganintelligentfilteringalgorithmforflyingadhocnetworksfanets |