Multi-Agent Deep Reinforcement Learning-Based Cooperative Spectrum Sensing With Upper Confidence Bound Exploration
In this paper, a multi-agent deep reinforcement learning method was adopted to realize cooperative spectrum sensing in cognitive radio networks. Each secondary user learns an efficient sensing strategy from the sensing results of some of the selected spectra to avoid interference to the primary user...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8811461/ |
| _version_ | 1798001989091590144 |
|---|---|
| author | Yu Zhang Peixiang Cai Changyong Pan Subing Zhang |
| author_facet | Yu Zhang Peixiang Cai Changyong Pan Subing Zhang |
| author_sort | Yu Zhang |
| collection | DOAJ |
| description | In this paper, a multi-agent deep reinforcement learning method was adopted to realize cooperative spectrum sensing in cognitive radio networks. Each secondary user learns an efficient sensing strategy from the sensing results of some of the selected spectra to avoid interference to the primary users and to coordinate with other secondary users. It is necessary to balance exploration and exploitation in the learning process when using deep reinforcement learning methods, helping explain that upper confidence bound with Hoeffding-style bonus has been adopted in this paper to improve the efficiency of exploration. The simulation results verify that the proposed algorithm, when compared with the conventional reinforcement learning methods with <inline-formula> <tex-math notation="LaTeX">$\varepsilon $ </tex-math></inline-formula>-greedy exploration, is much easier to achieve faster convergence speed and better reward performance. |
| first_indexed | 2024-04-11T11:44:59Z |
| format | Article |
| id | doaj.art-f3098dbf3a694f3abf4e9ba2f72c48fe |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T11:44:59Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-f3098dbf3a694f3abf4e9ba2f72c48fe2022-12-22T04:25:38ZengIEEEIEEE Access2169-35362019-01-01711889811890610.1109/ACCESS.2019.29371088811461Multi-Agent Deep Reinforcement Learning-Based Cooperative Spectrum Sensing With Upper Confidence Bound ExplorationYu Zhang0Peixiang Cai1https://orcid.org/0000-0002-0718-5496Changyong Pan2Subing Zhang3Department of Electronic Engineering, Tsinghua University, Beijing, ChinaDepartment of Electronic Engineering, Tsinghua University, Beijing, ChinaDepartment of Electronic Engineering, Tsinghua University, Beijing, ChinaChina Electronics Standardization Institute, Beijing, ChinaIn this paper, a multi-agent deep reinforcement learning method was adopted to realize cooperative spectrum sensing in cognitive radio networks. Each secondary user learns an efficient sensing strategy from the sensing results of some of the selected spectra to avoid interference to the primary users and to coordinate with other secondary users. It is necessary to balance exploration and exploitation in the learning process when using deep reinforcement learning methods, helping explain that upper confidence bound with Hoeffding-style bonus has been adopted in this paper to improve the efficiency of exploration. The simulation results verify that the proposed algorithm, when compared with the conventional reinforcement learning methods with <inline-formula> <tex-math notation="LaTeX">$\varepsilon $ </tex-math></inline-formula>-greedy exploration, is much easier to achieve faster convergence speed and better reward performance.https://ieeexplore.ieee.org/document/8811461/Cooperative spectrum sensingdeep reinforcement learningcognitive radioupper bound confidence |
| spellingShingle | Yu Zhang Peixiang Cai Changyong Pan Subing Zhang Multi-Agent Deep Reinforcement Learning-Based Cooperative Spectrum Sensing With Upper Confidence Bound Exploration IEEE Access Cooperative spectrum sensing deep reinforcement learning cognitive radio upper bound confidence |
| title | Multi-Agent Deep Reinforcement Learning-Based Cooperative Spectrum Sensing With Upper Confidence Bound Exploration |
| title_full | Multi-Agent Deep Reinforcement Learning-Based Cooperative Spectrum Sensing With Upper Confidence Bound Exploration |
| title_fullStr | Multi-Agent Deep Reinforcement Learning-Based Cooperative Spectrum Sensing With Upper Confidence Bound Exploration |
| title_full_unstemmed | Multi-Agent Deep Reinforcement Learning-Based Cooperative Spectrum Sensing With Upper Confidence Bound Exploration |
| title_short | Multi-Agent Deep Reinforcement Learning-Based Cooperative Spectrum Sensing With Upper Confidence Bound Exploration |
| title_sort | multi agent deep reinforcement learning based cooperative spectrum sensing with upper confidence bound exploration |
| topic | Cooperative spectrum sensing deep reinforcement learning cognitive radio upper bound confidence |
| url | https://ieeexplore.ieee.org/document/8811461/ |
| work_keys_str_mv | AT yuzhang multiagentdeepreinforcementlearningbasedcooperativespectrumsensingwithupperconfidenceboundexploration AT peixiangcai multiagentdeepreinforcementlearningbasedcooperativespectrumsensingwithupperconfidenceboundexploration AT changyongpan multiagentdeepreinforcementlearningbasedcooperativespectrumsensingwithupperconfidenceboundexploration AT subingzhang multiagentdeepreinforcementlearningbasedcooperativespectrumsensingwithupperconfidenceboundexploration |