Toward an Adaptive Threshold on Cooperative Bandwidth Management Based on Hierarchical Reinforcement Learning
With the increase in Internet of Things (IoT) devices and network communications, but with less bandwidth growth, the resulting constraints must be overcome. Due to the network complexity and uncertainty of emergency distribution parameters in smart environments, using predetermined rules seems illo...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/21/7053 |
| _version_ | 1797511816363900928 |
|---|---|
| author | Motahareh Mobasheri Yangwoo Kim Woongsup Kim |
| author_facet | Motahareh Mobasheri Yangwoo Kim Woongsup Kim |
| author_sort | Motahareh Mobasheri |
| collection | DOAJ |
| description | With the increase in Internet of Things (IoT) devices and network communications, but with less bandwidth growth, the resulting constraints must be overcome. Due to the network complexity and uncertainty of emergency distribution parameters in smart environments, using predetermined rules seems illogical. Reinforcement learning (RL), as a powerful machine learning approach, can handle such smart environments without a trainer or supervisor. Recently, we worked on bandwidth management in a smart environment with several fog fragments using limited shared bandwidth, where IoT devices may experience uncertain emergencies in terms of the time and sequence needed for more bandwidth for further higher-level communication. We introduced fog fragment cooperation using an RL approach under a predefined fixed threshold constraint. In this study, we promote this approach by removing the fixed level of restriction of the threshold through hierarchical reinforcement learning (HRL) and completing the cooperation qualification. At the first learning hierarchy level of the proposed approach, the best threshold level is learned over time, and the final results are used by the second learning hierarchy level, where the fog node learns the best device for helping an emergency device by temporarily lending the bandwidth. Although equipping the method to the adaptive threshold and restricting fog fragment cooperation make the learning procedure more difficult, the HRL approach increases the method’s efficiency in terms of time and performance. |
| first_indexed | 2024-03-10T05:53:21Z |
| format | Article |
| id | doaj.art-de66238ca697460091a3b3690707a228 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T05:53:21Z |
| publishDate | 2021-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-de66238ca697460091a3b3690707a2282023-11-22T21:35:53ZengMDPI AGSensors1424-82202021-10-012121705310.3390/s21217053Toward an Adaptive Threshold on Cooperative Bandwidth Management Based on Hierarchical Reinforcement LearningMotahareh Mobasheri0Yangwoo Kim1Woongsup Kim2Information and Communication Engineering Department, Dongguk University, Seoul 04620, KoreaInformation and Communication Engineering Department, Dongguk University, Seoul 04620, KoreaInformation and Communication Engineering Department, Dongguk University, Seoul 04620, KoreaWith the increase in Internet of Things (IoT) devices and network communications, but with less bandwidth growth, the resulting constraints must be overcome. Due to the network complexity and uncertainty of emergency distribution parameters in smart environments, using predetermined rules seems illogical. Reinforcement learning (RL), as a powerful machine learning approach, can handle such smart environments without a trainer or supervisor. Recently, we worked on bandwidth management in a smart environment with several fog fragments using limited shared bandwidth, where IoT devices may experience uncertain emergencies in terms of the time and sequence needed for more bandwidth for further higher-level communication. We introduced fog fragment cooperation using an RL approach under a predefined fixed threshold constraint. In this study, we promote this approach by removing the fixed level of restriction of the threshold through hierarchical reinforcement learning (HRL) and completing the cooperation qualification. At the first learning hierarchy level of the proposed approach, the best threshold level is learned over time, and the final results are used by the second learning hierarchy level, where the fog node learns the best device for helping an emergency device by temporarily lending the bandwidth. Although equipping the method to the adaptive threshold and restricting fog fragment cooperation make the learning procedure more difficult, the HRL approach increases the method’s efficiency in terms of time and performance.https://www.mdpi.com/1424-8220/21/21/7053internet of thingsfog computingfog fragment cooperationhierarchical reinforcement learning |
| spellingShingle | Motahareh Mobasheri Yangwoo Kim Woongsup Kim Toward an Adaptive Threshold on Cooperative Bandwidth Management Based on Hierarchical Reinforcement Learning Sensors internet of things fog computing fog fragment cooperation hierarchical reinforcement learning |
| title | Toward an Adaptive Threshold on Cooperative Bandwidth Management Based on Hierarchical Reinforcement Learning |
| title_full | Toward an Adaptive Threshold on Cooperative Bandwidth Management Based on Hierarchical Reinforcement Learning |
| title_fullStr | Toward an Adaptive Threshold on Cooperative Bandwidth Management Based on Hierarchical Reinforcement Learning |
| title_full_unstemmed | Toward an Adaptive Threshold on Cooperative Bandwidth Management Based on Hierarchical Reinforcement Learning |
| title_short | Toward an Adaptive Threshold on Cooperative Bandwidth Management Based on Hierarchical Reinforcement Learning |
| title_sort | toward an adaptive threshold on cooperative bandwidth management based on hierarchical reinforcement learning |
| topic | internet of things fog computing fog fragment cooperation hierarchical reinforcement learning |
| url | https://www.mdpi.com/1424-8220/21/21/7053 |
| work_keys_str_mv | AT motaharehmobasheri towardanadaptivethresholdoncooperativebandwidthmanagementbasedonhierarchicalreinforcementlearning AT yangwookim towardanadaptivethresholdoncooperativebandwidthmanagementbasedonhierarchicalreinforcementlearning AT woongsupkim towardanadaptivethresholdoncooperativebandwidthmanagementbasedonhierarchicalreinforcementlearning |