AVQBits—Adaptive Video Quality Model Based on Bitstream Information for Various Video Applications
The paper presents <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula>, a versatile, bitstream-based video quality model. It can be applied in several contexts such as video service monitoring, evaluation of video encoding quality, of...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9846967/ |
| _version_ | 1798040481250148352 |
|---|---|
| author | Rakesh Rao Ramachandra Rao Steve Goring Alexander Raake |
| author_facet | Rakesh Rao Ramachandra Rao Steve Goring Alexander Raake |
| author_sort | Rakesh Rao Ramachandra Rao |
| collection | DOAJ |
| description | The paper presents <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula>, a versatile, bitstream-based video quality model. It can be applied in several contexts such as video service monitoring, evaluation of video encoding quality, of gaming video QoE, and even of omnidirectional video quality. In the paper, it is shown that <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> predictions closely match video quality ratings obained in various subjective tests with human viewers, for videos up to 4K-UHD resolution (Ultra-High Definition, 3840 x 2180 pixels) and framerates up 120 fps. With the different variants of <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> presented in the paper, video quality can be monitored either at the client side, in the network or directly after encoding. The no-reference <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> model was developed for different video services and types of input data, reflecting the increasing popularity of Video-on-Demand services and widespread use of HTTP-based adaptive streaming. At its core, <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> encompasses the standardized ITU-T P.1204.3 model, with further model instances that can either have restricted or extended input information, depending on the application context. Four different instances of <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> are presented, that is, a Mode 3 model with full access to the bitstream, a Mode 0 variant using only metadata such as codec type, framerate, resoution and bitrate as input, a Mode 1 model using Mode 0 information and frame-type and -size information, and a Hybrid Mode 0 model that is based on Mode 0 metadata and the decoded video pixel information. The models are trained on the authors’ own AVT-PNATS-UHD-1 dataset described in the paper. All models show a highly competitive performance by using AVT-VQDB-UHD-1 as validation dataset, e.g., with the Mode 0 variant yielding a value of 0.890 Pearson Correlation, the Mode 1 model of 0.901, the hybrid no-reference mode 0 model of 0.928 and the model with full bitstream access of 0.942. In addition, all four <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> variants are evaluated when applying them out-of-the-box to different media formats such as 360° video, high framerate (HFR) content, or gaming videos. The analysis shows that the ITU-T P.1204.3 and Hybrid Mode 0 instances of <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> for the considered use-cases either perform on par with or better than even state-of-the-art full reference, pixel-based models. Furthermore, it is shown that the proposed Mode 0 and Mode 1 variants outperform commonly used no-reference models for the different application scopes. Also, a long-term integration model based on the standardized ITU-T P.1203.3 is presented to estimate ratings of overall audiovisual streaming Quality of Experience (QoE) for sessions of 30 s up to 5 min duration. In the paper, the <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> instances with their per-1-sec score output are evaluated as the video quality component of the proposed long-term integration model. All <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> variants as well as the long-term integration module are made publicly available for the community for further research. |
| first_indexed | 2024-04-11T22:08:12Z |
| format | Article |
| id | doaj.art-94f7723e375341d18febadd166e4a27c |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T22:08:12Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-94f7723e375341d18febadd166e4a27c2022-12-22T04:00:38ZengIEEEIEEE Access2169-35362022-01-0110803218035110.1109/ACCESS.2022.31955279846967AVQBits—Adaptive Video Quality Model Based on Bitstream Information for Various Video ApplicationsRakesh Rao Ramachandra Rao0https://orcid.org/0000-0002-7069-1543Steve Goring1https://orcid.org/0000-0001-6810-6969Alexander Raake2https://orcid.org/0000-0002-9357-1763Audiovisual Technology Group, Technische Universität Ilmenau, Ilmenau, GermanyAudiovisual Technology Group, Technische Universität Ilmenau, Ilmenau, GermanyAudiovisual Technology Group, Technische Universität Ilmenau, Ilmenau, GermanyThe paper presents <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula>, a versatile, bitstream-based video quality model. It can be applied in several contexts such as video service monitoring, evaluation of video encoding quality, of gaming video QoE, and even of omnidirectional video quality. In the paper, it is shown that <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> predictions closely match video quality ratings obained in various subjective tests with human viewers, for videos up to 4K-UHD resolution (Ultra-High Definition, 3840 x 2180 pixels) and framerates up 120 fps. With the different variants of <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> presented in the paper, video quality can be monitored either at the client side, in the network or directly after encoding. The no-reference <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> model was developed for different video services and types of input data, reflecting the increasing popularity of Video-on-Demand services and widespread use of HTTP-based adaptive streaming. At its core, <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> encompasses the standardized ITU-T P.1204.3 model, with further model instances that can either have restricted or extended input information, depending on the application context. Four different instances of <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> are presented, that is, a Mode 3 model with full access to the bitstream, a Mode 0 variant using only metadata such as codec type, framerate, resoution and bitrate as input, a Mode 1 model using Mode 0 information and frame-type and -size information, and a Hybrid Mode 0 model that is based on Mode 0 metadata and the decoded video pixel information. The models are trained on the authors’ own AVT-PNATS-UHD-1 dataset described in the paper. All models show a highly competitive performance by using AVT-VQDB-UHD-1 as validation dataset, e.g., with the Mode 0 variant yielding a value of 0.890 Pearson Correlation, the Mode 1 model of 0.901, the hybrid no-reference mode 0 model of 0.928 and the model with full bitstream access of 0.942. In addition, all four <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> variants are evaluated when applying them out-of-the-box to different media formats such as 360° video, high framerate (HFR) content, or gaming videos. The analysis shows that the ITU-T P.1204.3 and Hybrid Mode 0 instances of <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> for the considered use-cases either perform on par with or better than even state-of-the-art full reference, pixel-based models. Furthermore, it is shown that the proposed Mode 0 and Mode 1 variants outperform commonly used no-reference models for the different application scopes. Also, a long-term integration model based on the standardized ITU-T P.1203.3 is presented to estimate ratings of overall audiovisual streaming Quality of Experience (QoE) for sessions of 30 s up to 5 min duration. In the paper, the <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> instances with their per-1-sec score output are evaluated as the video quality component of the proposed long-term integration model. All <inline-formula> <tex-math notation="LaTeX">$AVQBits$ </tex-math></inline-formula> variants as well as the long-term integration module are made publicly available for the community for further research.https://ieeexplore.ieee.org/document/9846967/Bitstream video quality modelsquality of experience (QoE)quality assessmentHTTP-based adaptive streaming (HAS)hybrid modelsvideo quality |
| spellingShingle | Rakesh Rao Ramachandra Rao Steve Goring Alexander Raake AVQBits—Adaptive Video Quality Model Based on Bitstream Information for Various Video Applications IEEE Access Bitstream video quality models quality of experience (QoE) quality assessment HTTP-based adaptive streaming (HAS) hybrid models video quality |
| title | AVQBits—Adaptive Video Quality Model Based on Bitstream Information for Various Video Applications |
| title_full | AVQBits—Adaptive Video Quality Model Based on Bitstream Information for Various Video Applications |
| title_fullStr | AVQBits—Adaptive Video Quality Model Based on Bitstream Information for Various Video Applications |
| title_full_unstemmed | AVQBits—Adaptive Video Quality Model Based on Bitstream Information for Various Video Applications |
| title_short | AVQBits—Adaptive Video Quality Model Based on Bitstream Information for Various Video Applications |
| title_sort | avqbits x2014 adaptive video quality model based on bitstream information for various video applications |
| topic | Bitstream video quality models quality of experience (QoE) quality assessment HTTP-based adaptive streaming (HAS) hybrid models video quality |
| url | https://ieeexplore.ieee.org/document/9846967/ |
| work_keys_str_mv | AT rakeshraoramachandrarao avqbitsx2014adaptivevideoqualitymodelbasedonbitstreaminformationforvariousvideoapplications AT stevegoring avqbitsx2014adaptivevideoqualitymodelbasedonbitstreaminformationforvariousvideoapplications AT alexanderraake avqbitsx2014adaptivevideoqualitymodelbasedonbitstreaminformationforvariousvideoapplications |