Deep Reinforcement Learning Evolution Algorithm for Dynamic Antenna Control in Multi-Cell Configuration HAPS System
In this paper, we propose a novel Deep Reinforcement Learning Evolution Algorithm (DRLEA) method to control the antenna parameters of the High-Altitude Platform Station (HAPS) mobile to reduce the number of low-throughput users. Considering the random movement of the HAPS caused by the winds, the th...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Future Internet |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-5903/15/1/34 |
| _version_ | 1797442265390514176 |
|---|---|
| author | Siyuan Yang Mondher Bouazizi Tomoaki Ohtsuki Yohei Shibata Wataru Takabatake Kenji Hoshino Atsushi Nagate |
| author_facet | Siyuan Yang Mondher Bouazizi Tomoaki Ohtsuki Yohei Shibata Wataru Takabatake Kenji Hoshino Atsushi Nagate |
| author_sort | Siyuan Yang |
| collection | DOAJ |
| description | In this paper, we propose a novel Deep Reinforcement Learning Evolution Algorithm (DRLEA) method to control the antenna parameters of the High-Altitude Platform Station (HAPS) mobile to reduce the number of low-throughput users. Considering the random movement of the HAPS caused by the winds, the throughput of the users might decrease. Therefore, we propose a method that can dynamically adjust the antenna parameters based on the throughput of the users in the coverage area to reduce the number of low-throughput users by improving the users’ throughput. Different from other model-based reinforcement learning methods, such as the Deep <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi mathvariant="script">Q</mi></semantics></math></inline-formula> Network (DQN), the proposed method combines the Evolution Algorithm (EA) with Reinforcement Learning (RL) to avoid the sub-optimal solutions in each state. Moreover, we consider non-uniform user distribution scenarios, which are common in the real world, rather than ideal uniform user distribution scenarios. To evaluate the proposed method, we do the simulations under four different real user distribution scenarios and compare the proposed method with the conventional EA and RL methods. The simulation results show that the proposed method effectively reduces the number of low throughput users after the HAPS moves. |
| first_indexed | 2024-03-09T12:39:20Z |
| format | Article |
| id | doaj.art-e655c987d84b4879bc428929d38e46e8 |
| institution | Directory Open Access Journal |
| issn | 1999-5903 |
| language | English |
| last_indexed | 2024-03-09T12:39:20Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Future Internet |
| spelling | doaj.art-e655c987d84b4879bc428929d38e46e82023-11-30T22:20:51ZengMDPI AGFuture Internet1999-59032023-01-011513410.3390/fi15010034Deep Reinforcement Learning Evolution Algorithm for Dynamic Antenna Control in Multi-Cell Configuration HAPS SystemSiyuan Yang0Mondher Bouazizi1Tomoaki Ohtsuki2Yohei Shibata3Wataru Takabatake4Kenji Hoshino5Atsushi Nagate6Graduate School of Science and Technology, Keio University, Yokohama 223-8522, JapanDepartment of Information and Computer Science, Faculty of Science and Technology, Keio University, Yokohama 223-8522, JapanDepartment of Information and Computer Science, Faculty of Science and Technology, Keio University, Yokohama 223-8522, JapanSoftBank Corp. Technology Research Laboratory, Tokyo 135-0064, JapanSoftBank Corp. Technology Research Laboratory, Tokyo 135-0064, JapanSoftBank Corp. Technology Research Laboratory, Tokyo 135-0064, JapanSoftBank Corp. Technology Research Laboratory, Tokyo 135-0064, JapanIn this paper, we propose a novel Deep Reinforcement Learning Evolution Algorithm (DRLEA) method to control the antenna parameters of the High-Altitude Platform Station (HAPS) mobile to reduce the number of low-throughput users. Considering the random movement of the HAPS caused by the winds, the throughput of the users might decrease. Therefore, we propose a method that can dynamically adjust the antenna parameters based on the throughput of the users in the coverage area to reduce the number of low-throughput users by improving the users’ throughput. Different from other model-based reinforcement learning methods, such as the Deep <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi mathvariant="script">Q</mi></semantics></math></inline-formula> Network (DQN), the proposed method combines the Evolution Algorithm (EA) with Reinforcement Learning (RL) to avoid the sub-optimal solutions in each state. Moreover, we consider non-uniform user distribution scenarios, which are common in the real world, rather than ideal uniform user distribution scenarios. To evaluate the proposed method, we do the simulations under four different real user distribution scenarios and compare the proposed method with the conventional EA and RL methods. The simulation results show that the proposed method effectively reduces the number of low throughput users after the HAPS moves.https://www.mdpi.com/1999-5903/15/1/34HAPSantenna controlreinforcement learningevolution algorithm |
| spellingShingle | Siyuan Yang Mondher Bouazizi Tomoaki Ohtsuki Yohei Shibata Wataru Takabatake Kenji Hoshino Atsushi Nagate Deep Reinforcement Learning Evolution Algorithm for Dynamic Antenna Control in Multi-Cell Configuration HAPS System Future Internet HAPS antenna control reinforcement learning evolution algorithm |
| title | Deep Reinforcement Learning Evolution Algorithm for Dynamic Antenna Control in Multi-Cell Configuration HAPS System |
| title_full | Deep Reinforcement Learning Evolution Algorithm for Dynamic Antenna Control in Multi-Cell Configuration HAPS System |
| title_fullStr | Deep Reinforcement Learning Evolution Algorithm for Dynamic Antenna Control in Multi-Cell Configuration HAPS System |
| title_full_unstemmed | Deep Reinforcement Learning Evolution Algorithm for Dynamic Antenna Control in Multi-Cell Configuration HAPS System |
| title_short | Deep Reinforcement Learning Evolution Algorithm for Dynamic Antenna Control in Multi-Cell Configuration HAPS System |
| title_sort | deep reinforcement learning evolution algorithm for dynamic antenna control in multi cell configuration haps system |
| topic | HAPS antenna control reinforcement learning evolution algorithm |
| url | https://www.mdpi.com/1999-5903/15/1/34 |
| work_keys_str_mv | AT siyuanyang deepreinforcementlearningevolutionalgorithmfordynamicantennacontrolinmulticellconfigurationhapssystem AT mondherbouazizi deepreinforcementlearningevolutionalgorithmfordynamicantennacontrolinmulticellconfigurationhapssystem AT tomoakiohtsuki deepreinforcementlearningevolutionalgorithmfordynamicantennacontrolinmulticellconfigurationhapssystem AT yoheishibata deepreinforcementlearningevolutionalgorithmfordynamicantennacontrolinmulticellconfigurationhapssystem AT watarutakabatake deepreinforcementlearningevolutionalgorithmfordynamicantennacontrolinmulticellconfigurationhapssystem AT kenjihoshino deepreinforcementlearningevolutionalgorithmfordynamicantennacontrolinmulticellconfigurationhapssystem AT atsushinagate deepreinforcementlearningevolutionalgorithmfordynamicantennacontrolinmulticellconfigurationhapssystem |