Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications
Technology advancements in wireless communications and high-performance Extended Reality (XR) have empowered the developments of the Metaverse. The demand for the Metaverse applications and hence, real-time digital twinning of real-world scenes is increasing. Nevertheless, the replication of 2D phys...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/172579 |
| _version_ | 1811687407484928000 |
|---|---|
| author | Yu, Wenhan Chua, Terence Jie Zhao, Jun |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Yu, Wenhan Chua, Terence Jie Zhao, Jun |
| author_sort | Yu, Wenhan |
| collection | NTU |
| description | Technology advancements in wireless communications and high-performance Extended Reality (XR) have empowered the developments of the Metaverse. The demand for the Metaverse applications and hence, real-time digital twinning of real-world scenes is increasing. Nevertheless, the replication of 2D physical world images into 3D virtual objects is computationally intensive and requires computation offloading. The disparity in transmitted object dimension (2D as opposed to 3D) leads to asymmetric data sizes in uplink (UL) and downlink (DL). To ensure the reliability and low latency of the system, we consider an asynchronous joint UL-DL scenario where in the UL stage, the smaller data size of the physical world images captured by multiple extended reality users (XUs) will be uploaded to the Metaverse Console (MC) to be construed and rendered. In the DL stage, the larger-size 3D virtual objects need to be transmitted back to the XUs. We design a novel multi-agent reinforcement learning algorithm structure, namely Asynchronous Actors Hybrid Critic (AAHC), to optimize the decisions pertaining to computation offloading and channel assignment in the UL stage and optimize the DL transmission power in the DL stage. Extensive experiments demonstrate that compared to proposed baselines, AAHC obtains better solutions with satisfactory training time. |
| first_indexed | 2024-10-01T05:15:49Z |
| format | Journal Article |
| id | ntu-10356/172579 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:15:49Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | ntu-10356/1725792023-12-13T06:25:48Z Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications Yu, Wenhan Chua, Terence Jie Zhao, Jun School of Computer Science and Engineering Engineering::Computer science and engineering Reinforcement Learning Wireless Communications Technology advancements in wireless communications and high-performance Extended Reality (XR) have empowered the developments of the Metaverse. The demand for the Metaverse applications and hence, real-time digital twinning of real-world scenes is increasing. Nevertheless, the replication of 2D physical world images into 3D virtual objects is computationally intensive and requires computation offloading. The disparity in transmitted object dimension (2D as opposed to 3D) leads to asymmetric data sizes in uplink (UL) and downlink (DL). To ensure the reliability and low latency of the system, we consider an asynchronous joint UL-DL scenario where in the UL stage, the smaller data size of the physical world images captured by multiple extended reality users (XUs) will be uploaded to the Metaverse Console (MC) to be construed and rendered. In the DL stage, the larger-size 3D virtual objects need to be transmitted back to the XUs. We design a novel multi-agent reinforcement learning algorithm structure, namely Asynchronous Actors Hybrid Critic (AAHC), to optimize the decisions pertaining to computation offloading and channel assignment in the UL stage and optimize the DL transmission power in the DL stage. Extensive experiments demonstrate that compared to proposed baselines, AAHC obtains better solutions with satisfactory training time. Ministry of Education (MOE) Nanyang Technological University This work was supported in part by the Singapore Ministry of Education Academic Research Fund under Grant Tier 1 RG90/22, Grant RG97/20, Grant Tier 1 RG24/20, and Grant Tier 2 MOE2019-T2-1-176; and in part by the Nanyang Technological University-Wallenberg AI, Autonomous Systems and Software Program (WASP) Joint Project. 2023-12-13T06:25:48Z 2023-12-13T06:25:48Z 2023 Journal Article Yu, W., Chua, T. J. & Zhao, J. (2023). Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications. IEEE Journal On Selected Areas in Communications, 41(7), 2138-2157. https://dx.doi.org/10.1109/JSAC.2023.3280988 0733-8716 https://hdl.handle.net/10356/172579 10.1109/JSAC.2023.3280988 2-s2.0-85153893825 7 41 2138 2157 en RG90/22 RG97/20 RG24/20 MOE2019-T2-1-176 IEEE Journal on Selected Areas in Communications © 2023 IEEE. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering Reinforcement Learning Wireless Communications Yu, Wenhan Chua, Terence Jie Zhao, Jun Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications |
| title | Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications |
| title_full | Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications |
| title_fullStr | Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications |
| title_full_unstemmed | Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications |
| title_short | Asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications |
| title_sort | asynchronous hybrid reinforcement learning for latency and reliability optimization in the metaverse over wireless communications |
| topic | Engineering::Computer science and engineering Reinforcement Learning Wireless Communications |
| url | https://hdl.handle.net/10356/172579 |
| work_keys_str_mv | AT yuwenhan asynchronoushybridreinforcementlearningforlatencyandreliabilityoptimizationinthemetaverseoverwirelesscommunications AT chuaterencejie asynchronoushybridreinforcementlearningforlatencyandreliabilityoptimizationinthemetaverseoverwirelesscommunications AT zhaojun asynchronoushybridreinforcementlearningforlatencyandreliabilityoptimizationinthemetaverseoverwirelesscommunications |