A Software-Defined Directional Q-Learning Grid-Based Routing Platform and Its Two-Hop Trajectory-Based Routing Algorithm for Vehicular Ad Hoc Networks
Dealing with the packet-routing problem is challenging in the V2X (Vehicle-to-Everything) network environment, where it suffers from the high mobility of vehicles and varied vehicle density at different times. Many related studies have been proposed to apply artificial intelligence models, such as Q...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/21/8222 |
| _version_ | 1827645528502960128 |
|---|---|
| author | Chen-Pin Yang Chin-En Yen Ing-Chau Chang |
| author_facet | Chen-Pin Yang Chin-En Yen Ing-Chau Chang |
| author_sort | Chen-Pin Yang |
| collection | DOAJ |
| description | Dealing with the packet-routing problem is challenging in the V2X (Vehicle-to-Everything) network environment, where it suffers from the high mobility of vehicles and varied vehicle density at different times. Many related studies have been proposed to apply artificial intelligence models, such as Q-learning, which is a well-known reinforcement learning model, to analyze the historical trajectory data of vehicles and to further design an efficient packet-routing algorithm for V2X. In order to reduce the number of Q-tables generated by Q-learning, grid-based routing algorithms such as the QGrid have been proposed accordingly to divide the entire network environment into equal grids. This paper focuses on improving the defects of these grid-based routing algorithms, which only consider the vehicle density of each grid in Q-learning. Hence, we propose a Software-Defined Directional QGrid (SD-QGrid) routing platform in this paper. By deploying an SDN Control Node (CN) to perform centralized control for V2X, the SD-QGrid considers the directionality from the source to the destination, real-time positions and historical trajectory records between the adjacent grids of all vehicles. The SD-QGrid further proposes the flows of the offline Q-learning training process and the online routing decision process. The two-hop trajectory-based routing (THTR) algorithm, which depends on the source–destination directionality and the movement direction of the vehicle for the next two grids, is proposed as a vehicle node to forward its packets to the best next-hop neighbor node in real time. Finally, we use the real vehicle trajectory data of Taipei City to conduct extensive simulation experiments with respect to four transmission parameters. The simulation results prove that the SD-QGrid achieved an over 10% improvement in the average packet delivery ratio and an over 25% reduction in the average end-to-end delay at the cost of less than 2% in average overhead, compared with two well-known Q-learning grid-based routing algorithms. |
| first_indexed | 2024-03-09T18:41:30Z |
| format | Article |
| id | doaj.art-d4d0ad38b2854c35bdfd50fdf6e8a54b |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T18:41:30Z |
| publishDate | 2022-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-d4d0ad38b2854c35bdfd50fdf6e8a54b2023-11-24T06:44:39ZengMDPI AGSensors1424-82202022-10-012221822210.3390/s22218222A Software-Defined Directional Q-Learning Grid-Based Routing Platform and Its Two-Hop Trajectory-Based Routing Algorithm for Vehicular Ad Hoc NetworksChen-Pin Yang0Chin-En Yen1Ing-Chau Chang2Department of Computer Science and Information Engineering, National Changhua University of Education, Changhua 50007, TaiwanDepartment of Early Childhood Development and Education, Chaoyang University of Technology, Taichung 41349, TaiwanDepartment of Computer Science and Information Engineering, National Changhua University of Education, Changhua 50007, TaiwanDealing with the packet-routing problem is challenging in the V2X (Vehicle-to-Everything) network environment, where it suffers from the high mobility of vehicles and varied vehicle density at different times. Many related studies have been proposed to apply artificial intelligence models, such as Q-learning, which is a well-known reinforcement learning model, to analyze the historical trajectory data of vehicles and to further design an efficient packet-routing algorithm for V2X. In order to reduce the number of Q-tables generated by Q-learning, grid-based routing algorithms such as the QGrid have been proposed accordingly to divide the entire network environment into equal grids. This paper focuses on improving the defects of these grid-based routing algorithms, which only consider the vehicle density of each grid in Q-learning. Hence, we propose a Software-Defined Directional QGrid (SD-QGrid) routing platform in this paper. By deploying an SDN Control Node (CN) to perform centralized control for V2X, the SD-QGrid considers the directionality from the source to the destination, real-time positions and historical trajectory records between the adjacent grids of all vehicles. The SD-QGrid further proposes the flows of the offline Q-learning training process and the online routing decision process. The two-hop trajectory-based routing (THTR) algorithm, which depends on the source–destination directionality and the movement direction of the vehicle for the next two grids, is proposed as a vehicle node to forward its packets to the best next-hop neighbor node in real time. Finally, we use the real vehicle trajectory data of Taipei City to conduct extensive simulation experiments with respect to four transmission parameters. The simulation results prove that the SD-QGrid achieved an over 10% improvement in the average packet delivery ratio and an over 25% reduction in the average end-to-end delay at the cost of less than 2% in average overhead, compared with two well-known Q-learning grid-based routing algorithms.https://www.mdpi.com/1424-8220/22/21/8222vehicle to everything (V2X)reinforcement learningQ-learningSoftware-Defined Network (SDN)Software-Defined Directional QGrid (SD-QGrid)source–destination directionality |
| spellingShingle | Chen-Pin Yang Chin-En Yen Ing-Chau Chang A Software-Defined Directional Q-Learning Grid-Based Routing Platform and Its Two-Hop Trajectory-Based Routing Algorithm for Vehicular Ad Hoc Networks Sensors vehicle to everything (V2X) reinforcement learning Q-learning Software-Defined Network (SDN) Software-Defined Directional QGrid (SD-QGrid) source–destination directionality |
| title | A Software-Defined Directional Q-Learning Grid-Based Routing Platform and Its Two-Hop Trajectory-Based Routing Algorithm for Vehicular Ad Hoc Networks |
| title_full | A Software-Defined Directional Q-Learning Grid-Based Routing Platform and Its Two-Hop Trajectory-Based Routing Algorithm for Vehicular Ad Hoc Networks |
| title_fullStr | A Software-Defined Directional Q-Learning Grid-Based Routing Platform and Its Two-Hop Trajectory-Based Routing Algorithm for Vehicular Ad Hoc Networks |
| title_full_unstemmed | A Software-Defined Directional Q-Learning Grid-Based Routing Platform and Its Two-Hop Trajectory-Based Routing Algorithm for Vehicular Ad Hoc Networks |
| title_short | A Software-Defined Directional Q-Learning Grid-Based Routing Platform and Its Two-Hop Trajectory-Based Routing Algorithm for Vehicular Ad Hoc Networks |
| title_sort | software defined directional q learning grid based routing platform and its two hop trajectory based routing algorithm for vehicular ad hoc networks |
| topic | vehicle to everything (V2X) reinforcement learning Q-learning Software-Defined Network (SDN) Software-Defined Directional QGrid (SD-QGrid) source–destination directionality |
| url | https://www.mdpi.com/1424-8220/22/21/8222 |
| work_keys_str_mv | AT chenpinyang asoftwaredefineddirectionalqlearninggridbasedroutingplatformanditstwohoptrajectorybasedroutingalgorithmforvehicularadhocnetworks AT chinenyen asoftwaredefineddirectionalqlearninggridbasedroutingplatformanditstwohoptrajectorybasedroutingalgorithmforvehicularadhocnetworks AT ingchauchang asoftwaredefineddirectionalqlearninggridbasedroutingplatformanditstwohoptrajectorybasedroutingalgorithmforvehicularadhocnetworks AT chenpinyang softwaredefineddirectionalqlearninggridbasedroutingplatformanditstwohoptrajectorybasedroutingalgorithmforvehicularadhocnetworks AT chinenyen softwaredefineddirectionalqlearninggridbasedroutingplatformanditstwohoptrajectorybasedroutingalgorithmforvehicularadhocnetworks AT ingchauchang softwaredefineddirectionalqlearninggridbasedroutingplatformanditstwohoptrajectorybasedroutingalgorithmforvehicularadhocnetworks |