Transfer restless multi-armed bandit policy for energy-efficient heterogeneous cellular network
Abstract This paper proposes a learning policy to improve the energy efficiency (EE) of heterogeneous cellular networks. The combination of active and inactive base stations (BS) that allows for maximizing EE is identified as a combinatorial learning problem and requires high computational complexit...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2019-10-01
|
| Series: | EURASIP Journal on Advances in Signal Processing |
| Subjects: | |
| Online Access: | http://link.springer.com/article/10.1186/s13634-019-0637-1 |
| _version_ | 1818450024919465984 |
|---|---|
| author | Navikkumar Modi Philippe Mary Christophe Moy |
| author_facet | Navikkumar Modi Philippe Mary Christophe Moy |
| author_sort | Navikkumar Modi |
| collection | DOAJ |
| description | Abstract This paper proposes a learning policy to improve the energy efficiency (EE) of heterogeneous cellular networks. The combination of active and inactive base stations (BS) that allows for maximizing EE is identified as a combinatorial learning problem and requires high computational complexity as well as a large signaling overhead. This paper aims at presenting a learning policy that dynamically switches a BS to ON or OFF status in order to follow the traffic load variation during the day. The network traffic load is represented as a Markov decision process, and we propose a modified upper confidence bound algorithm based on restless Markov multi-armed bandit framework for the BS switching operation. Moreover, to cope with initial reward loss and to speed up the convergence of the learning algorithm, the transfer learning concept is adapted to our algorithm in order to benefit from the transferred knowledge observed in historical periods from the same region. Based on our previous work, a convergence theorem is provided for the proposed policy. Extensive simulations demonstrate that the proposed algorithms follow the traffic load variation during the day and contribute to a performance jump-start in EE improvement under various practical traffic load profiles. It also demonstrates that proposed schemes can significantly reduce the total energy consumption of cellular network, e.g., up to 70% potential energy savings based on a real traffic profile. |
| first_indexed | 2024-12-14T20:44:44Z |
| format | Article |
| id | doaj.art-0c8a2ce59d7d45d6bb89b15509ffd617 |
| institution | Directory Open Access Journal |
| issn | 1687-6180 |
| language | English |
| last_indexed | 2024-12-14T20:44:44Z |
| publishDate | 2019-10-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | EURASIP Journal on Advances in Signal Processing |
| spelling | doaj.art-0c8a2ce59d7d45d6bb89b15509ffd6172022-12-21T22:48:07ZengSpringerOpenEURASIP Journal on Advances in Signal Processing1687-61802019-10-012019111910.1186/s13634-019-0637-1Transfer restless multi-armed bandit policy for energy-efficient heterogeneous cellular networkNavikkumar Modi0Philippe Mary1Christophe Moy2Brussels Airport CompanyUniv. Rennes, INSA de Rennes, CNRS, IETR - UMR 6164Univ. Rennes, CNRS, IETR - UMR 6164Abstract This paper proposes a learning policy to improve the energy efficiency (EE) of heterogeneous cellular networks. The combination of active and inactive base stations (BS) that allows for maximizing EE is identified as a combinatorial learning problem and requires high computational complexity as well as a large signaling overhead. This paper aims at presenting a learning policy that dynamically switches a BS to ON or OFF status in order to follow the traffic load variation during the day. The network traffic load is represented as a Markov decision process, and we propose a modified upper confidence bound algorithm based on restless Markov multi-armed bandit framework for the BS switching operation. Moreover, to cope with initial reward loss and to speed up the convergence of the learning algorithm, the transfer learning concept is adapted to our algorithm in order to benefit from the transferred knowledge observed in historical periods from the same region. Based on our previous work, a convergence theorem is provided for the proposed policy. Extensive simulations demonstrate that the proposed algorithms follow the traffic load variation during the day and contribute to a performance jump-start in EE improvement under various practical traffic load profiles. It also demonstrates that proposed schemes can significantly reduce the total energy consumption of cellular network, e.g., up to 70% potential energy savings based on a real traffic profile.http://link.springer.com/article/10.1186/s13634-019-0637-1Energy efficiencyGreen cellular networksUpper confidence boundReinforcement learningTransfer learningMulti-armed bandit |
| spellingShingle | Navikkumar Modi Philippe Mary Christophe Moy Transfer restless multi-armed bandit policy for energy-efficient heterogeneous cellular network EURASIP Journal on Advances in Signal Processing Energy efficiency Green cellular networks Upper confidence bound Reinforcement learning Transfer learning Multi-armed bandit |
| title | Transfer restless multi-armed bandit policy for energy-efficient heterogeneous cellular network |
| title_full | Transfer restless multi-armed bandit policy for energy-efficient heterogeneous cellular network |
| title_fullStr | Transfer restless multi-armed bandit policy for energy-efficient heterogeneous cellular network |
| title_full_unstemmed | Transfer restless multi-armed bandit policy for energy-efficient heterogeneous cellular network |
| title_short | Transfer restless multi-armed bandit policy for energy-efficient heterogeneous cellular network |
| title_sort | transfer restless multi armed bandit policy for energy efficient heterogeneous cellular network |
| topic | Energy efficiency Green cellular networks Upper confidence bound Reinforcement learning Transfer learning Multi-armed bandit |
| url | http://link.springer.com/article/10.1186/s13634-019-0637-1 |
| work_keys_str_mv | AT navikkumarmodi transferrestlessmultiarmedbanditpolicyforenergyefficientheterogeneouscellularnetwork AT philippemary transferrestlessmultiarmedbanditpolicyforenergyefficientheterogeneouscellularnetwork AT christophemoy transferrestlessmultiarmedbanditpolicyforenergyefficientheterogeneouscellularnetwork |