Multi‐agent reinforcement learning based transmission scheme for IRS‐assisted multi‐UAV systems
Abstract In this paper, a transmission scheme based on multi‐agent reinforcement learning for intelligent reflecting surface (IRS)‐assisted multiple unmanned aerial vehicles (UAVs) systems is proposed. The proposed scheme is based on reinforcement learning and alternating optimization algorithm, whi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-10-01
|
| Series: | IET Communications |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/cmu2.12674 |
| _version_ | 1797661482006085632 |
|---|---|
| author | Yumo Mei Chen Liu Yunchao Song Ge Wang Huibin Liang |
| author_facet | Yumo Mei Chen Liu Yunchao Song Ge Wang Huibin Liang |
| author_sort | Yumo Mei |
| collection | DOAJ |
| description | Abstract In this paper, a transmission scheme based on multi‐agent reinforcement learning for intelligent reflecting surface (IRS)‐assisted multiple unmanned aerial vehicles (UAVs) systems is proposed. The proposed scheme is based on reinforcement learning and alternating optimization algorithm, which can effectively improve communication quality and ensure fairness. The scheme is divided into two parts. In the first part, the multi‐UAV cooperation problem is modeled as a markov decision process. The objective of each UAV is to maximize the minimum user channel gain. To achieve stable strategies for all agents, the Multi‐agent Deep Deterministic Policy Gradient (MADDPG) algorithm is applied to train UAVs trajectories to reach the Nash equilibrium. The MADDPG algorithm is centralized trained at the base station and executed in a distributed manner by each UAV, ensuring efficient and effective coordination among agents. In the second part, an alternating optimization algorithm is formulated to optimize active and passive beamforming. Considering the non‐convexity of the fairness objective, by using auxiliary variables and semi‐definite relaxation method, the problem of maximizing the minimum user achievable rate is transformed into a feasibility problem. Simulation results show that the proposed scheme can effectively train UAVs trajectories and improve the communication performance of all users fairly. |
| first_indexed | 2024-03-11T18:45:12Z |
| format | Article |
| id | doaj.art-e476781f97d24a14ab805ca8b15e1926 |
| institution | Directory Open Access Journal |
| issn | 1751-8628 1751-8636 |
| language | English |
| last_indexed | 2024-03-11T18:45:12Z |
| publishDate | 2023-10-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Communications |
| spelling | doaj.art-e476781f97d24a14ab805ca8b15e19262023-10-12T05:26:30ZengWileyIET Communications1751-86281751-86362023-10-0117172019202910.1049/cmu2.12674Multi‐agent reinforcement learning based transmission scheme for IRS‐assisted multi‐UAV systemsYumo Mei0Chen Liu1Yunchao Song2Ge Wang3Huibin Liang4College of Electronic and Optical Engineering & College of Flexible Electronics (Future Technology) Nanjing University of Posts and Telecommunications Nanjing Jiangsu ChinaCollege of Electronic and Optical Engineering & College of Flexible Electronics (Future Technology) Nanjing University of Posts and Telecommunications Nanjing Jiangsu ChinaCollege of Electronic and Optical Engineering & College of Flexible Electronics (Future Technology) Nanjing University of Posts and Telecommunications Nanjing Jiangsu ChinaCollege of Electronic and Optical Engineering & College of Flexible Electronics (Future Technology) Nanjing University of Posts and Telecommunications Nanjing Jiangsu ChinaCollege of Electronic and Optical Engineering & College of Flexible Electronics (Future Technology) Nanjing University of Posts and Telecommunications Nanjing Jiangsu ChinaAbstract In this paper, a transmission scheme based on multi‐agent reinforcement learning for intelligent reflecting surface (IRS)‐assisted multiple unmanned aerial vehicles (UAVs) systems is proposed. The proposed scheme is based on reinforcement learning and alternating optimization algorithm, which can effectively improve communication quality and ensure fairness. The scheme is divided into two parts. In the first part, the multi‐UAV cooperation problem is modeled as a markov decision process. The objective of each UAV is to maximize the minimum user channel gain. To achieve stable strategies for all agents, the Multi‐agent Deep Deterministic Policy Gradient (MADDPG) algorithm is applied to train UAVs trajectories to reach the Nash equilibrium. The MADDPG algorithm is centralized trained at the base station and executed in a distributed manner by each UAV, ensuring efficient and effective coordination among agents. In the second part, an alternating optimization algorithm is formulated to optimize active and passive beamforming. Considering the non‐convexity of the fairness objective, by using auxiliary variables and semi‐definite relaxation method, the problem of maximizing the minimum user achievable rate is transformed into a feasibility problem. Simulation results show that the proposed scheme can effectively train UAVs trajectories and improve the communication performance of all users fairly.https://doi.org/10.1049/cmu2.12674multi‐agent systemsMIMO communication |
| spellingShingle | Yumo Mei Chen Liu Yunchao Song Ge Wang Huibin Liang Multi‐agent reinforcement learning based transmission scheme for IRS‐assisted multi‐UAV systems IET Communications multi‐agent systems MIMO communication |
| title | Multi‐agent reinforcement learning based transmission scheme for IRS‐assisted multi‐UAV systems |
| title_full | Multi‐agent reinforcement learning based transmission scheme for IRS‐assisted multi‐UAV systems |
| title_fullStr | Multi‐agent reinforcement learning based transmission scheme for IRS‐assisted multi‐UAV systems |
| title_full_unstemmed | Multi‐agent reinforcement learning based transmission scheme for IRS‐assisted multi‐UAV systems |
| title_short | Multi‐agent reinforcement learning based transmission scheme for IRS‐assisted multi‐UAV systems |
| title_sort | multi agent reinforcement learning based transmission scheme for irs assisted multi uav systems |
| topic | multi‐agent systems MIMO communication |
| url | https://doi.org/10.1049/cmu2.12674 |
| work_keys_str_mv | AT yumomei multiagentreinforcementlearningbasedtransmissionschemeforirsassistedmultiuavsystems AT chenliu multiagentreinforcementlearningbasedtransmissionschemeforirsassistedmultiuavsystems AT yunchaosong multiagentreinforcementlearningbasedtransmissionschemeforirsassistedmultiuavsystems AT gewang multiagentreinforcementlearningbasedtransmissionschemeforirsassistedmultiuavsystems AT huibinliang multiagentreinforcementlearningbasedtransmissionschemeforirsassistedmultiuavsystems |