Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT Networks
To support dramatically increasing services from internet of thing (IoT) devices with the sporadic and fluctuated generation of short packet traffic, this paper investigates joint dynamic time division duplexing (TDD) and radio resource control (RRC) connection management in a single-cell massive Io...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10452324/ |
| _version_ | 1797243364211425280 |
|---|---|
| author | Jaeeun Park Joohyung Lee Daejin Kim Jun Kyun Choi |
| author_facet | Jaeeun Park Joohyung Lee Daejin Kim Jun Kyun Choi |
| author_sort | Jaeeun Park |
| collection | DOAJ |
| description | To support dramatically increasing services from internet of thing (IoT) devices with the sporadic and fluctuated generation of short packet traffic, this paper investigates joint dynamic time division duplexing (TDD) and radio resource control (RRC) connection management in a single-cell massive IoT network. Specifically, under the grant-free transmission incurring packet collision, this study models the factors affecting the time resource utilization (TRU) and energy consumption of IoT devices as a comprehensive system utility and further formulates the problem as a decision-making process aiming for balancing the long-term average TRU and energy consumption. To address the formulated problem, based on the deep reinforcement learning framework, this paper designs an experience-driven joint dynamic TDD and RRC connection management scheme that intelligently i) determines the TDD configuration based on the most recent downlink (DL)/uplink (UL) traffic demands and ii) adjusts the RRC state of each IoT device to control the maximum number of transmitting IoT devices. Finally, trace-driven simulation results demonstrate that the proposed scheme outperforms existing benchmarks, such as Static TDD and Dynamic TDD, in terms of transmission success ratio difference (TSRD) (up to 89% reduction), time resource utilization (TRU) (up to <inline-formula> <tex-math notation="LaTeX">$17\times $ </tex-math></inline-formula> increase), and energy consumption (up to 70% reduction) of IoT devices. |
| first_indexed | 2024-04-24T18:53:56Z |
| format | Article |
| id | doaj.art-5cf93784fb1e46be8f24a473e5af1e6c |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-24T18:53:56Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-5cf93784fb1e46be8f24a473e5af1e6c2024-03-26T17:46:44ZengIEEEIEEE Access2169-35362024-01-0112349733499210.1109/ACCESS.2024.337116910452324Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT NetworksJaeeun Park0https://orcid.org/0009-0001-5251-7672Joohyung Lee1https://orcid.org/0000-0003-1102-3905Daejin Kim2https://orcid.org/0000-0001-5452-7906Jun Kyun Choi3https://orcid.org/0000-0002-0151-1297School of Information and Communication Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South KoreaSchool of Computing, Gachon University, Seongnam-si, South KoreaSamsung Electronics, Suwon, South KoreaSchool of Information and Communication Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South KoreaTo support dramatically increasing services from internet of thing (IoT) devices with the sporadic and fluctuated generation of short packet traffic, this paper investigates joint dynamic time division duplexing (TDD) and radio resource control (RRC) connection management in a single-cell massive IoT network. Specifically, under the grant-free transmission incurring packet collision, this study models the factors affecting the time resource utilization (TRU) and energy consumption of IoT devices as a comprehensive system utility and further formulates the problem as a decision-making process aiming for balancing the long-term average TRU and energy consumption. To address the formulated problem, based on the deep reinforcement learning framework, this paper designs an experience-driven joint dynamic TDD and RRC connection management scheme that intelligently i) determines the TDD configuration based on the most recent downlink (DL)/uplink (UL) traffic demands and ii) adjusts the RRC state of each IoT device to control the maximum number of transmitting IoT devices. Finally, trace-driven simulation results demonstrate that the proposed scheme outperforms existing benchmarks, such as Static TDD and Dynamic TDD, in terms of transmission success ratio difference (TSRD) (up to 89% reduction), time resource utilization (TRU) (up to <inline-formula> <tex-math notation="LaTeX">$17\times $ </tex-math></inline-formula> increase), and energy consumption (up to 70% reduction) of IoT devices.https://ieeexplore.ieee.org/document/10452324/Internet of Thingstime division duplexinggrant-free transmissionradio resource controldeep reinforcement learning |
| spellingShingle | Jaeeun Park Joohyung Lee Daejin Kim Jun Kyun Choi Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT Networks IEEE Access Internet of Things time division duplexing grant-free transmission radio resource control deep reinforcement learning |
| title | Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT Networks |
| title_full | Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT Networks |
| title_fullStr | Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT Networks |
| title_full_unstemmed | Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT Networks |
| title_short | Deep Reinforcement Learning Driven Joint Dynamic TDD and RRC Connection Management Scheme in Massive IoT Networks |
| title_sort | deep reinforcement learning driven joint dynamic tdd and rrc connection management scheme in massive iot networks |
| topic | Internet of Things time division duplexing grant-free transmission radio resource control deep reinforcement learning |
| url | https://ieeexplore.ieee.org/document/10452324/ |
| work_keys_str_mv | AT jaeeunpark deepreinforcementlearningdrivenjointdynamictddandrrcconnectionmanagementschemeinmassiveiotnetworks AT joohyunglee deepreinforcementlearningdrivenjointdynamictddandrrcconnectionmanagementschemeinmassiveiotnetworks AT daejinkim deepreinforcementlearningdrivenjointdynamictddandrrcconnectionmanagementschemeinmassiveiotnetworks AT junkyunchoi deepreinforcementlearningdrivenjointdynamictddandrrcconnectionmanagementschemeinmassiveiotnetworks |