Handover Decision Making for Dense HetNets: A Reinforcement Learning Approach
In this paper, we consider the problem of decision making in the context of a dense heterogeneous network with a macro base station and multiple small base stations. We propose a deep Q-learning based algorithm that efficiently minimizes the overall energy consumption by taking into account both the...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10064273/ |
| _version_ | 1827988590569717760 |
|---|---|
| author | Yujae Song Sung Hoon Lim Sang-Woon Jeon |
| author_facet | Yujae Song Sung Hoon Lim Sang-Woon Jeon |
| author_sort | Yujae Song |
| collection | DOAJ |
| description | In this paper, we consider the problem of decision making in the context of a dense heterogeneous network with a macro base station and multiple small base stations. We propose a deep Q-learning based algorithm that efficiently minimizes the overall energy consumption by taking into account both the energy consumption from transmission and overheads, and various network information such as channel conditions and causal association information. The proposed algorithm is designed based on the centralized training with decentralized execution (CTDE) framework in which a centralized training agent manages the replay buffer for training its deep Q-network by gathering state, action, and reward information reported from the distributed agents that execute the actions. We perform several numerical evaluations and demonstrate that the proposed algorithm provides significant energy savings over other contemporary mechanisms depending on overhead costs, especially when additional energy consumption is required for handover procedure. |
| first_indexed | 2024-04-10T00:05:55Z |
| format | Article |
| id | doaj.art-b8d2fedfc7c24302a641a5b17c5386b9 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-10T00:05:55Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-b8d2fedfc7c24302a641a5b17c5386b92023-03-16T23:00:31ZengIEEEIEEE Access2169-35362023-01-0111247372475110.1109/ACCESS.2023.325455710064273Handover Decision Making for Dense HetNets: A Reinforcement Learning ApproachYujae Song0https://orcid.org/0000-0001-6346-7271Sung Hoon Lim1https://orcid.org/0000-0002-7792-2033Sang-Woon Jeon2https://orcid.org/0000-0002-0199-2254Department of Robotics Engineering, Yeungnam University, Gyeongsan, South KoreaSchool of Information Sciences, Hallym University, Chuncheon, South KoreaDepartment of Electrical and Electronic Engineering, Hanyang University, Ansan, South KoreaIn this paper, we consider the problem of decision making in the context of a dense heterogeneous network with a macro base station and multiple small base stations. We propose a deep Q-learning based algorithm that efficiently minimizes the overall energy consumption by taking into account both the energy consumption from transmission and overheads, and various network information such as channel conditions and causal association information. The proposed algorithm is designed based on the centralized training with decentralized execution (CTDE) framework in which a centralized training agent manages the replay buffer for training its deep Q-network by gathering state, action, and reward information reported from the distributed agents that execute the actions. We perform several numerical evaluations and demonstrate that the proposed algorithm provides significant energy savings over other contemporary mechanisms depending on overhead costs, especially when additional energy consumption is required for handover procedure.https://ieeexplore.ieee.org/document/10064273/Deep learningcentralized training decentralized executionenergy minimizationheterogeneous networksload balancingreinforcement learning |
| spellingShingle | Yujae Song Sung Hoon Lim Sang-Woon Jeon Handover Decision Making for Dense HetNets: A Reinforcement Learning Approach IEEE Access Deep learning centralized training decentralized execution energy minimization heterogeneous networks load balancing reinforcement learning |
| title | Handover Decision Making for Dense HetNets: A Reinforcement Learning Approach |
| title_full | Handover Decision Making for Dense HetNets: A Reinforcement Learning Approach |
| title_fullStr | Handover Decision Making for Dense HetNets: A Reinforcement Learning Approach |
| title_full_unstemmed | Handover Decision Making for Dense HetNets: A Reinforcement Learning Approach |
| title_short | Handover Decision Making for Dense HetNets: A Reinforcement Learning Approach |
| title_sort | handover decision making for dense hetnets a reinforcement learning approach |
| topic | Deep learning centralized training decentralized execution energy minimization heterogeneous networks load balancing reinforcement learning |
| url | https://ieeexplore.ieee.org/document/10064273/ |
| work_keys_str_mv | AT yujaesong handoverdecisionmakingfordensehetnetsareinforcementlearningapproach AT sunghoonlim handoverdecisionmakingfordensehetnetsareinforcementlearningapproach AT sangwoonjeon handoverdecisionmakingfordensehetnetsareinforcementlearningapproach |