Deep Learning Approach for Seamless Navigation in Multi-View Streaming Applications
Quality of Experience (QoE) in multi-view streaming systems is known to be severely affected by the latency associated with view-switching procedures. Anticipating the navigation intentions of the viewer on the multi-view scene could provide the means to greatly reduce such latency. The research wor...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10235963/ |
| _version_ | 1797689565901750272 |
|---|---|
| author | Tiago S. Costa Paula Viana Maria T. Andrade |
| author_facet | Tiago S. Costa Paula Viana Maria T. Andrade |
| author_sort | Tiago S. Costa |
| collection | DOAJ |
| description | Quality of Experience (QoE) in multi-view streaming systems is known to be severely affected by the latency associated with view-switching procedures. Anticipating the navigation intentions of the viewer on the multi-view scene could provide the means to greatly reduce such latency. The research work presented in this article builds on this premise by proposing a new predictive view-selection mechanism. A VGG16-inspired Convolutional Neural Network (CNN) is used to identify the viewer’s focus of attention and determine which views would be most suited to be presented in the brief term, i.e., the near-term viewing intentions. This way, those views can be locally buffered before they are actually needed. To this aim, two datasets were used to evaluate the prediction performance and impact on latency, in particular when compared to the solution implemented in the previous version of our multi-view streaming system. Results obtained with this work translate into a generalized improvement in perceived QoE. A significant reduction in latency during view-switching procedures was effectively achieved. Moreover, results also demonstrated that the prediction of the user’s visual interest was achieved with a high level of accuracy. An experimental platform was also established on which future predictive models can be integrated and compared with previously implemented models. |
| first_indexed | 2024-03-12T01:47:25Z |
| format | Article |
| id | doaj.art-66e21586208447d882511bf5ea60d884 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-12T01:47:25Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-66e21586208447d882511bf5ea60d8842023-09-08T23:01:32ZengIEEEIEEE Access2169-35362023-01-0111938839389710.1109/ACCESS.2023.331082210235963Deep Learning Approach for Seamless Navigation in Multi-View Streaming ApplicationsTiago S. Costa0https://orcid.org/0000-0002-3778-8773Paula Viana1https://orcid.org/0000-0001-8447-2360Maria T. Andrade2https://orcid.org/0000-0002-1363-5027Centre for Telecommunications and Multimedia, INESC TEC, Porto, PortugalCentre for Telecommunications and Multimedia, INESC TEC, Porto, PortugalCentre for Telecommunications and Multimedia, INESC TEC, Porto, PortugalQuality of Experience (QoE) in multi-view streaming systems is known to be severely affected by the latency associated with view-switching procedures. Anticipating the navigation intentions of the viewer on the multi-view scene could provide the means to greatly reduce such latency. The research work presented in this article builds on this premise by proposing a new predictive view-selection mechanism. A VGG16-inspired Convolutional Neural Network (CNN) is used to identify the viewer’s focus of attention and determine which views would be most suited to be presented in the brief term, i.e., the near-term viewing intentions. This way, those views can be locally buffered before they are actually needed. To this aim, two datasets were used to evaluate the prediction performance and impact on latency, in particular when compared to the solution implemented in the previous version of our multi-view streaming system. Results obtained with this work translate into a generalized improvement in perceived QoE. A significant reduction in latency during view-switching procedures was effectively achieved. Moreover, results also demonstrated that the prediction of the user’s visual interest was achieved with a high level of accuracy. An experimental platform was also established on which future predictive models can be integrated and compared with previously implemented models.https://ieeexplore.ieee.org/document/10235963/Multimediastreamingmulti-viewfocus-of-attentiondeep learning |
| spellingShingle | Tiago S. Costa Paula Viana Maria T. Andrade Deep Learning Approach for Seamless Navigation in Multi-View Streaming Applications IEEE Access Multimedia streaming multi-view focus-of-attention deep learning |
| title | Deep Learning Approach for Seamless Navigation in Multi-View Streaming Applications |
| title_full | Deep Learning Approach for Seamless Navigation in Multi-View Streaming Applications |
| title_fullStr | Deep Learning Approach for Seamless Navigation in Multi-View Streaming Applications |
| title_full_unstemmed | Deep Learning Approach for Seamless Navigation in Multi-View Streaming Applications |
| title_short | Deep Learning Approach for Seamless Navigation in Multi-View Streaming Applications |
| title_sort | deep learning approach for seamless navigation in multi view streaming applications |
| topic | Multimedia streaming multi-view focus-of-attention deep learning |
| url | https://ieeexplore.ieee.org/document/10235963/ |
| work_keys_str_mv | AT tiagoscosta deeplearningapproachforseamlessnavigationinmultiviewstreamingapplications AT paulaviana deeplearningapproachforseamlessnavigationinmultiviewstreamingapplications AT mariatandrade deeplearningapproachforseamlessnavigationinmultiviewstreamingapplications |