Multi-Agent Reinforcement-Learning-Based Time-Slotted Channel Hopping Medium Access Control Scheduling Scheme
Time-slotted channel hopping (TSCH) is a medium access control technology that realizes collision-free wireless network communication by coordinating the media access time and channel of network devices. Although existing TSCH schedulers have suitable application scenarios for each, they are less ve...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9144568/ |
| _version_ | 1818936169027600384 |
|---|---|
| author | Huiung Park Haeyong Kim Seon-Tae Kim Pyeongsoo Mah |
| author_facet | Huiung Park Haeyong Kim Seon-Tae Kim Pyeongsoo Mah |
| author_sort | Huiung Park |
| collection | DOAJ |
| description | Time-slotted channel hopping (TSCH) is a medium access control technology that realizes collision-free wireless network communication by coordinating the media access time and channel of network devices. Although existing TSCH schedulers have suitable application scenarios for each, they are less versatile. Scheduling without collisions inevitably lowers the throughput, whereas contention-based scheduling achieves high-throughput but it may induces to frequent collisions in densely deployed networks. Therefore, a TSCH scheduler that can be used universally, regardless of the topology and data collection characteristics of the application scenario, is required to overcome these shortcomings. To this end, a multi-agent reinforcement learning (RL)-based TSCH scheduling scheme that allows contention but minimizes collisions is proposed in this study. RL is a machine-learning method that gradually improves actions to solve problems. One specific RL method, Q-Learning (QL), was used in the scheme to enable the TSCH scheduler to become a QL agent that learns the best transmission slot. To improve the QL performance, reward functions tailored for the TSCH scheduler were developed. Because the QL agent runs on multiple nodes concurrently, changes in the TSCH schedule of one node also affect the performance of the TSCH schedules of other nodes. The use of action peeking is proposed to overcome this non-stationarity problem in a multi-agent environment. The experimental results indicate that the TSCH scheduler consistently performs well in various types of applications, compared to other schedulers. |
| first_indexed | 2024-12-20T05:31:47Z |
| format | Article |
| id | doaj.art-0f7513d61d3143968c1ee199aee9bdcb |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-20T05:31:47Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-0f7513d61d3143968c1ee199aee9bdcb2022-12-21T19:51:42ZengIEEEIEEE Access2169-35362020-01-01813972713973610.1109/ACCESS.2020.30105759144568Multi-Agent Reinforcement-Learning-Based Time-Slotted Channel Hopping Medium Access Control Scheduling SchemeHuiung Park0https://orcid.org/0000-0002-3322-7629Haeyong Kim1https://orcid.org/0000-0003-1039-0949Seon-Tae Kim2Pyeongsoo Mah3Electronics and Telecommunication Research Institute (ETRI), Daejeon, South KoreaElectronics and Telecommunication Research Institute (ETRI), Daejeon, South KoreaElectronics and Telecommunication Research Institute (ETRI), Daejeon, South KoreaElectronics and Telecommunication Research Institute (ETRI), Daejeon, South KoreaTime-slotted channel hopping (TSCH) is a medium access control technology that realizes collision-free wireless network communication by coordinating the media access time and channel of network devices. Although existing TSCH schedulers have suitable application scenarios for each, they are less versatile. Scheduling without collisions inevitably lowers the throughput, whereas contention-based scheduling achieves high-throughput but it may induces to frequent collisions in densely deployed networks. Therefore, a TSCH scheduler that can be used universally, regardless of the topology and data collection characteristics of the application scenario, is required to overcome these shortcomings. To this end, a multi-agent reinforcement learning (RL)-based TSCH scheduling scheme that allows contention but minimizes collisions is proposed in this study. RL is a machine-learning method that gradually improves actions to solve problems. One specific RL method, Q-Learning (QL), was used in the scheme to enable the TSCH scheduler to become a QL agent that learns the best transmission slot. To improve the QL performance, reward functions tailored for the TSCH scheduler were developed. Because the QL agent runs on multiple nodes concurrently, changes in the TSCH schedule of one node also affect the performance of the TSCH schedules of other nodes. The use of action peeking is proposed to overcome this non-stationarity problem in a multi-agent environment. The experimental results indicate that the TSCH scheduler consistently performs well in various types of applications, compared to other schedulers.https://ieeexplore.ieee.org/document/9144568/Internet of Things (IoT)time-slotted channel hopping (TSCH) schedulingwireless sensor networks |
| spellingShingle | Huiung Park Haeyong Kim Seon-Tae Kim Pyeongsoo Mah Multi-Agent Reinforcement-Learning-Based Time-Slotted Channel Hopping Medium Access Control Scheduling Scheme IEEE Access Internet of Things (IoT) time-slotted channel hopping (TSCH) scheduling wireless sensor networks |
| title | Multi-Agent Reinforcement-Learning-Based Time-Slotted Channel Hopping Medium Access Control Scheduling Scheme |
| title_full | Multi-Agent Reinforcement-Learning-Based Time-Slotted Channel Hopping Medium Access Control Scheduling Scheme |
| title_fullStr | Multi-Agent Reinforcement-Learning-Based Time-Slotted Channel Hopping Medium Access Control Scheduling Scheme |
| title_full_unstemmed | Multi-Agent Reinforcement-Learning-Based Time-Slotted Channel Hopping Medium Access Control Scheduling Scheme |
| title_short | Multi-Agent Reinforcement-Learning-Based Time-Slotted Channel Hopping Medium Access Control Scheduling Scheme |
| title_sort | multi agent reinforcement learning based time slotted channel hopping medium access control scheduling scheme |
| topic | Internet of Things (IoT) time-slotted channel hopping (TSCH) scheduling wireless sensor networks |
| url | https://ieeexplore.ieee.org/document/9144568/ |
| work_keys_str_mv | AT huiungpark multiagentreinforcementlearningbasedtimeslottedchannelhoppingmediumaccesscontrolschedulingscheme AT haeyongkim multiagentreinforcementlearningbasedtimeslottedchannelhoppingmediumaccesscontrolschedulingscheme AT seontaekim multiagentreinforcementlearningbasedtimeslottedchannelhoppingmediumaccesscontrolschedulingscheme AT pyeongsoomah multiagentreinforcementlearningbasedtimeslottedchannelhoppingmediumaccesscontrolschedulingscheme |