A Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning for LEO Satellite Networks
Low Earth Orbit (LEO) satellite networks can provide complete connectivity and worldwide data transmission capability for the internet of things. However, arbitrary flow arrival and uneven traffic load among areas bring about unbalanced traffic distribution over the LEO constellation. Therefore, the...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-08-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/8/9/920 |
| _version_ | 1798041899156635648 |
|---|---|
| author | Cheng Wang Huiwen Wang Weidong Wang |
| author_facet | Cheng Wang Huiwen Wang Weidong Wang |
| author_sort | Cheng Wang |
| collection | DOAJ |
| description | Low Earth Orbit (LEO) satellite networks can provide complete connectivity and worldwide data transmission capability for the internet of things. However, arbitrary flow arrival and uneven traffic load among areas bring about unbalanced traffic distribution over the LEO constellation. Therefore, the routing strategy in LEO networks should have the ability to adjust routing paths based on changes in network status adaptively. In this paper, we propose a Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning (DRL-THSA) for LEO satellite networks. In this strategy, each node only needs to obtain the link state within the range of two-hop neighbors, and the optimal next-hop node can be output. The link state is divided into three levels, and the traffic forwarding strategy for each level is proposed, which allows DRL-THSA to cope with link outage or congestion. The Double-Deep Q Network (DDQN) is proposed in DRL-THSA to figure out the optional next hop by inputting the two-hops link states. The DDQN is analyzed from three aspects: model setting, training process and running process. The effectiveness of DRL-THSA, in terms of end-to-end delay, throughput, and packet drop rate, is verified via a set of simulations using the Network Simulator 3 (NS3). |
| first_indexed | 2024-04-11T22:28:01Z |
| format | Article |
| id | doaj.art-f86fd6fbd57f46728015574fac7ea14a |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-04-11T22:28:01Z |
| publishDate | 2019-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-f86fd6fbd57f46728015574fac7ea14a2022-12-22T03:59:36ZengMDPI AGElectronics2079-92922019-08-018992010.3390/electronics8090920electronics8090920A Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning for LEO Satellite NetworksCheng Wang0Huiwen Wang1Weidong Wang2School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, ChinaSchool of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, ChinaSchool of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, ChinaLow Earth Orbit (LEO) satellite networks can provide complete connectivity and worldwide data transmission capability for the internet of things. However, arbitrary flow arrival and uneven traffic load among areas bring about unbalanced traffic distribution over the LEO constellation. Therefore, the routing strategy in LEO networks should have the ability to adjust routing paths based on changes in network status adaptively. In this paper, we propose a Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning (DRL-THSA) for LEO satellite networks. In this strategy, each node only needs to obtain the link state within the range of two-hop neighbors, and the optimal next-hop node can be output. The link state is divided into three levels, and the traffic forwarding strategy for each level is proposed, which allows DRL-THSA to cope with link outage or congestion. The Double-Deep Q Network (DDQN) is proposed in DRL-THSA to figure out the optional next hop by inputting the two-hops link states. The DDQN is analyzed from three aspects: model setting, training process and running process. The effectiveness of DRL-THSA, in terms of end-to-end delay, throughput, and packet drop rate, is verified via a set of simulations using the Network Simulator 3 (NS3).https://www.mdpi.com/2079-9292/8/9/920LEO satellite networkssatellite routingstate awarevirtual nodedeep reinforcement leaning |
| spellingShingle | Cheng Wang Huiwen Wang Weidong Wang A Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning for LEO Satellite Networks Electronics LEO satellite networks satellite routing state aware virtual node deep reinforcement leaning |
| title | A Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning for LEO Satellite Networks |
| title_full | A Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning for LEO Satellite Networks |
| title_fullStr | A Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning for LEO Satellite Networks |
| title_full_unstemmed | A Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning for LEO Satellite Networks |
| title_short | A Two-Hops State-Aware Routing Strategy Based on Deep Reinforcement Learning for LEO Satellite Networks |
| title_sort | two hops state aware routing strategy based on deep reinforcement learning for leo satellite networks |
| topic | LEO satellite networks satellite routing state aware virtual node deep reinforcement leaning |
| url | https://www.mdpi.com/2079-9292/8/9/920 |
| work_keys_str_mv | AT chengwang atwohopsstateawareroutingstrategybasedondeepreinforcementlearningforleosatellitenetworks AT huiwenwang atwohopsstateawareroutingstrategybasedondeepreinforcementlearningforleosatellitenetworks AT weidongwang atwohopsstateawareroutingstrategybasedondeepreinforcementlearningforleosatellitenetworks AT chengwang twohopsstateawareroutingstrategybasedondeepreinforcementlearningforleosatellitenetworks AT huiwenwang twohopsstateawareroutingstrategybasedondeepreinforcementlearningforleosatellitenetworks AT weidongwang twohopsstateawareroutingstrategybasedondeepreinforcementlearningforleosatellitenetworks |