Reinforcement Learning-Based Multihop Relaying: A Decentralized Q-Learning Approach
Conventional optimization-based relay selection for multihop networks cannot resolve the conflict between performance and cost. The optimal selection policy is centralized and requires local channel state information (CSI) of all hops, leading to high computational complexity and signaling overhead....
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-10-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/23/10/1310 |
| _version_ | 1797514670603501568 |
|---|---|
| author | Xiaowei Wang Xin Wang |
| author_facet | Xiaowei Wang Xin Wang |
| author_sort | Xiaowei Wang |
| collection | DOAJ |
| description | Conventional optimization-based relay selection for multihop networks cannot resolve the conflict between performance and cost. The optimal selection policy is centralized and requires local channel state information (CSI) of all hops, leading to high computational complexity and signaling overhead. Other optimization-based decentralized policies cause non-negligible performance loss. In this paper, we exploit the benefits of reinforcement learning in relay selection for multihop clustered networks and aim to achieve high performance with limited costs. Multihop relay selection problem is modeled as Markov decision process (MDP) and solved by a decentralized Q-learning scheme with rectified update function. Simulation results show that this scheme achieves near-optimal average end-to-end (E2E) rate. Cost analysis reveals that it also reduces computation complexity and signaling overhead compared with the optimal scheme. |
| first_indexed | 2024-03-10T06:34:53Z |
| format | Article |
| id | doaj.art-a02b63d4e0944163a6ce650d1c0347e2 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T06:34:53Z |
| publishDate | 2021-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-a02b63d4e0944163a6ce650d1c0347e22023-11-22T18:11:03ZengMDPI AGEntropy1099-43002021-10-012310131010.3390/e23101310Reinforcement Learning-Based Multihop Relaying: A Decentralized Q-Learning ApproachXiaowei Wang0Xin Wang1College of Information Engineering, Shanghai Maritime University, Shanghai 201306, ChinaCollege of Information Engineering, Shanghai Maritime University, Shanghai 201306, ChinaConventional optimization-based relay selection for multihop networks cannot resolve the conflict between performance and cost. The optimal selection policy is centralized and requires local channel state information (CSI) of all hops, leading to high computational complexity and signaling overhead. Other optimization-based decentralized policies cause non-negligible performance loss. In this paper, we exploit the benefits of reinforcement learning in relay selection for multihop clustered networks and aim to achieve high performance with limited costs. Multihop relay selection problem is modeled as Markov decision process (MDP) and solved by a decentralized Q-learning scheme with rectified update function. Simulation results show that this scheme achieves near-optimal average end-to-end (E2E) rate. Cost analysis reveals that it also reduces computation complexity and signaling overhead compared with the optimal scheme.https://www.mdpi.com/1099-4300/23/10/1310reinforcement learningQ-learningmultihop networkrelay selection |
| spellingShingle | Xiaowei Wang Xin Wang Reinforcement Learning-Based Multihop Relaying: A Decentralized Q-Learning Approach Entropy reinforcement learning Q-learning multihop network relay selection |
| title | Reinforcement Learning-Based Multihop Relaying: A Decentralized Q-Learning Approach |
| title_full | Reinforcement Learning-Based Multihop Relaying: A Decentralized Q-Learning Approach |
| title_fullStr | Reinforcement Learning-Based Multihop Relaying: A Decentralized Q-Learning Approach |
| title_full_unstemmed | Reinforcement Learning-Based Multihop Relaying: A Decentralized Q-Learning Approach |
| title_short | Reinforcement Learning-Based Multihop Relaying: A Decentralized Q-Learning Approach |
| title_sort | reinforcement learning based multihop relaying a decentralized q learning approach |
| topic | reinforcement learning Q-learning multihop network relay selection |
| url | https://www.mdpi.com/1099-4300/23/10/1310 |
| work_keys_str_mv | AT xiaoweiwang reinforcementlearningbasedmultihoprelayingadecentralizedqlearningapproach AT xinwang reinforcementlearningbasedmultihoprelayingadecentralizedqlearningapproach |