Viewport-Dependent Delivery for Conversational Immersive Video
Real-time immersive video has been an area of interest for the research and standardization community over the past few years. It has gained particular importance in the era of 5G services. 3GPP (Third Generation Partnership Project) Release 17 includes Immersive teleconferencing for Multimedia Tele...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9964346/ |
| _version_ | 1811197854804344832 |
|---|---|
| author | Saba Ahsan Sujeet Mate Igor D. D. Curcio Alireza Aminlou Yu You Emre B. Aksu Miska M. Hannuksela |
| author_facet | Saba Ahsan Sujeet Mate Igor D. D. Curcio Alireza Aminlou Yu You Emre B. Aksu Miska M. Hannuksela |
| author_sort | Saba Ahsan |
| collection | DOAJ |
| description | Real-time immersive video has been an area of interest for the research and standardization community over the past few years. It has gained particular importance in the era of 5G services. 3GPP (Third Generation Partnership Project) Release 17 includes Immersive teleconferencing for Multimedia Telephony services over IP Multimedia System. Viewport-dependent delivery (VDD) is a mechanism for bandwidth savings by assigning more bits to the part of the 360-degree video currently in the viewport and fewer or no bits to the areas of the 360-degree video that a user does not currently view. The technique has been extensively studied in the context of adaptive HTTP streaming of immersive video, where the 360-degree video is pre-encoded in different versions, and the appropriate version is selected and requested by the viewer’s client application, depending on their current viewport orientation. However, conversational video has more stringent latency requirements than HTTP streaming; the time from capture to rendering is in the order of a few milliseconds for a good user experience. Conversational immersive video also allows adapting the encoding during the media delivery session according to the user viewport and network bandwidth characteristics. This paper explores tile-based and viewport region encoding for VDD and how they apply to conversational video. In particular, we explore the challenges related to the absence of periodic intra-coded frames in conversational video. Based on simulation results, we show that VDD methods may not always be a performant alternative to viewport-independent delivery in conversational use cases. We also discuss possible encoder optimization that would improve the VDD performance. |
| first_indexed | 2024-04-12T01:20:02Z |
| format | Article |
| id | doaj.art-b8e59e685da8474a83a23134631968f1 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T01:20:02Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-b8e59e685da8474a83a23134631968f12022-12-22T03:53:48ZengIEEEIEEE Access2169-35362022-01-011012953912955110.1109/ACCESS.2022.32252319964346Viewport-Dependent Delivery for Conversational Immersive VideoSaba Ahsan0https://orcid.org/0000-0002-5028-6161Sujeet Mate1Igor D. D. Curcio2https://orcid.org/0000-0002-2234-933XAlireza Aminlou3Yu You4https://orcid.org/0000-0002-5273-1740Emre B. Aksu5Miska M. Hannuksela6https://orcid.org/0000-0003-3405-0850Nokia Technologies, Espoo, FinlandNokia Technologies, Tampere, FinlandNokia Technologies, Tampere, FinlandNokia Technologies, Tampere, FinlandNokia Technologies, Tampere, FinlandNokia Technologies, Tampere, FinlandNokia Technologies, Tampere, FinlandReal-time immersive video has been an area of interest for the research and standardization community over the past few years. It has gained particular importance in the era of 5G services. 3GPP (Third Generation Partnership Project) Release 17 includes Immersive teleconferencing for Multimedia Telephony services over IP Multimedia System. Viewport-dependent delivery (VDD) is a mechanism for bandwidth savings by assigning more bits to the part of the 360-degree video currently in the viewport and fewer or no bits to the areas of the 360-degree video that a user does not currently view. The technique has been extensively studied in the context of adaptive HTTP streaming of immersive video, where the 360-degree video is pre-encoded in different versions, and the appropriate version is selected and requested by the viewer’s client application, depending on their current viewport orientation. However, conversational video has more stringent latency requirements than HTTP streaming; the time from capture to rendering is in the order of a few milliseconds for a good user experience. Conversational immersive video also allows adapting the encoding during the media delivery session according to the user viewport and network bandwidth characteristics. This paper explores tile-based and viewport region encoding for VDD and how they apply to conversational video. In particular, we explore the challenges related to the absence of periodic intra-coded frames in conversational video. Based on simulation results, we show that VDD methods may not always be a performant alternative to viewport-independent delivery in conversational use cases. We also discuss possible encoder optimization that would improve the VDD performance.https://ieeexplore.ieee.org/document/9964346/360-degree videoconversational VRimmersive mediaviewport-dependent deliveryviewport-dependent processingvirtual reality |
| spellingShingle | Saba Ahsan Sujeet Mate Igor D. D. Curcio Alireza Aminlou Yu You Emre B. Aksu Miska M. Hannuksela Viewport-Dependent Delivery for Conversational Immersive Video IEEE Access 360-degree video conversational VR immersive media viewport-dependent delivery viewport-dependent processing virtual reality |
| title | Viewport-Dependent Delivery for Conversational Immersive Video |
| title_full | Viewport-Dependent Delivery for Conversational Immersive Video |
| title_fullStr | Viewport-Dependent Delivery for Conversational Immersive Video |
| title_full_unstemmed | Viewport-Dependent Delivery for Conversational Immersive Video |
| title_short | Viewport-Dependent Delivery for Conversational Immersive Video |
| title_sort | viewport dependent delivery for conversational immersive video |
| topic | 360-degree video conversational VR immersive media viewport-dependent delivery viewport-dependent processing virtual reality |
| url | https://ieeexplore.ieee.org/document/9964346/ |
| work_keys_str_mv | AT sabaahsan viewportdependentdeliveryforconversationalimmersivevideo AT sujeetmate viewportdependentdeliveryforconversationalimmersivevideo AT igorddcurcio viewportdependentdeliveryforconversationalimmersivevideo AT alirezaaminlou viewportdependentdeliveryforconversationalimmersivevideo AT yuyou viewportdependentdeliveryforconversationalimmersivevideo AT emrebaksu viewportdependentdeliveryforconversationalimmersivevideo AT miskamhannuksela viewportdependentdeliveryforconversationalimmersivevideo |