UAV-Assisted Fair Communication for Mobile Networks: A Multi-Agent Deep Reinforcement Learning Approach
Unmanned Aerial Vehicles (UAVs) can be employed as low-altitude aerial base stations (UAV-BSs) to provide communication services for ground users (GUs). However, most existing works mainly focus on optimizing coverage and maximizing throughput, without considering the fairness of the GUs in communic...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/22/5662 |
| _version_ | 1797464111742713856 |
|---|---|
| author | Yi Zhou Zhanqi Jin Huaguang Shi Zhangyun Wang Ning Lu Fuqiang Liu |
| author_facet | Yi Zhou Zhanqi Jin Huaguang Shi Zhangyun Wang Ning Lu Fuqiang Liu |
| author_sort | Yi Zhou |
| collection | DOAJ |
| description | Unmanned Aerial Vehicles (UAVs) can be employed as low-altitude aerial base stations (UAV-BSs) to provide communication services for ground users (GUs). However, most existing works mainly focus on optimizing coverage and maximizing throughput, without considering the fairness of the GUs in communication services. This may result in certain GUs being underserviced by UAV-BSs in pursuit of maximum throughput. In this paper, we study the problem of UAV-assisted communication with the consideration of user fairness. We first design a Ratio Fair (RF) metric by weighting fairness and throughput to evaluate the tradeoff between fairness and communication efficiency when UAV-BSs serve GUs. The problem is formulated as a mixed-integer non-convex optimization problem based on the RF metric and we propose a UAV-Assisted Fair Communication (UAFC) algorithm based on multi-agent deep reinforcement learning to maximize the fair throughput of the system. The UAFC algorithm comprehensively considers fair throughput, UAV-BSs coverage, and flight status to design a reasonable reward function. In addition, the UAFC algorithm establishes an information sharing mechanism based on gated functions by sharing neural networks, which effectively reduces the distributed decision-making uncertainty of UAV-BSs. To reduce the impact of state dimension imbalance on the convergence of the algorithm, we design a new state decomposing and coupling actor network architecture. Simulation results show that the proposed UAFC algorithm increases fair throughput by 5.62%, 26.57% and fair index by 1.99%, 13.82% compared to the MATD3 and MADDPG algorithms, respectively. Meanwhile, UAFC can also meet energy consumption limitation and network connectivity requirement. |
| first_indexed | 2024-03-09T18:02:19Z |
| format | Article |
| id | doaj.art-be08ad23196845d6858082dd3b86ba4f |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-09T18:02:19Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-be08ad23196845d6858082dd3b86ba4f2023-11-24T09:48:26ZengMDPI AGRemote Sensing2072-42922022-11-011422566210.3390/rs14225662UAV-Assisted Fair Communication for Mobile Networks: A Multi-Agent Deep Reinforcement Learning ApproachYi Zhou0Zhanqi Jin1Huaguang Shi2Zhangyun Wang3Ning Lu4Fuqiang Liu5School of Artificial Intelligence, Henan University, Zhengzhou 450046, ChinaSchool of Artificial Intelligence, Henan University, Zhengzhou 450046, ChinaSchool of Artificial Intelligence, Henan University, Zhengzhou 450046, ChinaSchool of Artificial Intelligence, Henan University, Zhengzhou 450046, ChinaDepartment of Electrical and Computer Engineering, Queen’s University, Kingston, ON K7L 3N6, CanadaCollege of Electronic and Information Engineering, Tongji University, Shanghai 201804, ChinaUnmanned Aerial Vehicles (UAVs) can be employed as low-altitude aerial base stations (UAV-BSs) to provide communication services for ground users (GUs). However, most existing works mainly focus on optimizing coverage and maximizing throughput, without considering the fairness of the GUs in communication services. This may result in certain GUs being underserviced by UAV-BSs in pursuit of maximum throughput. In this paper, we study the problem of UAV-assisted communication with the consideration of user fairness. We first design a Ratio Fair (RF) metric by weighting fairness and throughput to evaluate the tradeoff between fairness and communication efficiency when UAV-BSs serve GUs. The problem is formulated as a mixed-integer non-convex optimization problem based on the RF metric and we propose a UAV-Assisted Fair Communication (UAFC) algorithm based on multi-agent deep reinforcement learning to maximize the fair throughput of the system. The UAFC algorithm comprehensively considers fair throughput, UAV-BSs coverage, and flight status to design a reasonable reward function. In addition, the UAFC algorithm establishes an information sharing mechanism based on gated functions by sharing neural networks, which effectively reduces the distributed decision-making uncertainty of UAV-BSs. To reduce the impact of state dimension imbalance on the convergence of the algorithm, we design a new state decomposing and coupling actor network architecture. Simulation results show that the proposed UAFC algorithm increases fair throughput by 5.62%, 26.57% and fair index by 1.99%, 13.82% compared to the MATD3 and MADDPG algorithms, respectively. Meanwhile, UAFC can also meet energy consumption limitation and network connectivity requirement.https://www.mdpi.com/2072-4292/14/22/5662Unmanned Aerial Vehicles (UAVs)Multi-Agent Deep Reinforcement Learning (MADRL)fair communicationinformation sharing mechanism |
| spellingShingle | Yi Zhou Zhanqi Jin Huaguang Shi Zhangyun Wang Ning Lu Fuqiang Liu UAV-Assisted Fair Communication for Mobile Networks: A Multi-Agent Deep Reinforcement Learning Approach Remote Sensing Unmanned Aerial Vehicles (UAVs) Multi-Agent Deep Reinforcement Learning (MADRL) fair communication information sharing mechanism |
| title | UAV-Assisted Fair Communication for Mobile Networks: A Multi-Agent Deep Reinforcement Learning Approach |
| title_full | UAV-Assisted Fair Communication for Mobile Networks: A Multi-Agent Deep Reinforcement Learning Approach |
| title_fullStr | UAV-Assisted Fair Communication for Mobile Networks: A Multi-Agent Deep Reinforcement Learning Approach |
| title_full_unstemmed | UAV-Assisted Fair Communication for Mobile Networks: A Multi-Agent Deep Reinforcement Learning Approach |
| title_short | UAV-Assisted Fair Communication for Mobile Networks: A Multi-Agent Deep Reinforcement Learning Approach |
| title_sort | uav assisted fair communication for mobile networks a multi agent deep reinforcement learning approach |
| topic | Unmanned Aerial Vehicles (UAVs) Multi-Agent Deep Reinforcement Learning (MADRL) fair communication information sharing mechanism |
| url | https://www.mdpi.com/2072-4292/14/22/5662 |
| work_keys_str_mv | AT yizhou uavassistedfaircommunicationformobilenetworksamultiagentdeepreinforcementlearningapproach AT zhanqijin uavassistedfaircommunicationformobilenetworksamultiagentdeepreinforcementlearningapproach AT huaguangshi uavassistedfaircommunicationformobilenetworksamultiagentdeepreinforcementlearningapproach AT zhangyunwang uavassistedfaircommunicationformobilenetworksamultiagentdeepreinforcementlearningapproach AT ninglu uavassistedfaircommunicationformobilenetworksamultiagentdeepreinforcementlearningapproach AT fuqiangliu uavassistedfaircommunicationformobilenetworksamultiagentdeepreinforcementlearningapproach |