A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC
This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Open Journal of Signal Processing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9465693/ |
| _version_ | 1818939157340225536 |
|---|---|
| author | Fatemeh Nasiri Wassim Hamidouche Luce Morin Nicolas Dhollande Gildas Cocherel |
| author_facet | Fatemeh Nasiri Wassim Hamidouche Luce Morin Nicolas Dhollande Gildas Cocherel |
| author_sort | Fatemeh Nasiri |
| collection | DOAJ |
| description | This paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore, to retain a low memory requirement for the proposed method, one model is used for all Quantization Parameters (QPs) with a Quantization Parameter (QP)-map, which is also shared between luma and chroma components. In addition to the Post Processing (PP) approach, the In-Loop Filtering (ILF) codec integration has also been considered, where the characteristics of the Group of Pictures (GoP) are taken into account to boost the performance. The proposed CNN-based Quality Enhancement (QE) framework has been implemented on top of the Versatile Video Coding (VVC) Test Model (VTM-10). Experiments show that the prediction-aware aspect of the proposed method improves the coding efficiency gain of the default CNN-based QE method by 1.52%, in terms of BD-BR, at the same network complexity compared to the default CNN-based QE filter. |
| first_indexed | 2024-12-20T06:19:17Z |
| format | Article |
| id | doaj.art-4e9ab58aedb141868e87c6b63b9793e5 |
| institution | Directory Open Access Journal |
| issn | 2644-1322 |
| language | English |
| last_indexed | 2024-12-20T06:19:17Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Signal Processing |
| spelling | doaj.art-4e9ab58aedb141868e87c6b63b9793e52022-12-21T19:50:27ZengIEEEIEEE Open Journal of Signal Processing2644-13222021-01-01246648310.1109/OJSP.2021.30925989465693A CNN-Based Prediction-Aware Quality Enhancement Framework for VVCFatemeh Nasiri0https://orcid.org/0000-0003-0640-8149Wassim Hamidouche1https://orcid.org/0000-0002-0143-1756Luce Morin2Nicolas Dhollande3Gildas Cocherel4University of Rennes 1, INSA Rennes, CNRS, IETR - UMR 6164, Rennes, FranceUniversity of Rennes 1, INSA Rennes, CNRS, IETR - UMR 6164, Rennes, FranceUniversity of Rennes 1, INSA Rennes, CNRS, IETR - UMR 6164, Rennes, FranceAVIWEST, Parc Edonia, Batiment X1, Rue de la Terre de Feu, Saint-Grégoire, FranceAVIWEST, Parc Edonia, Batiment X1, Rue de la Terre de Feu, Saint-Grégoire, FranceThis paper presents a framework for Convolutional Neural Network (CNN)-based quality enhancement task, by taking advantage of coding information in the compressed video signal. The motivation is that normative decisions made by the encoder can significantly impact the type and strength of artifacts in the decoded images. In this paper, the main focus has been put on decisions defining the prediction signal in intra and inter frames. This information has been used in the training phase as well as input to help the process of learning artifacts that are specific to each coding type. Furthermore, to retain a low memory requirement for the proposed method, one model is used for all Quantization Parameters (QPs) with a Quantization Parameter (QP)-map, which is also shared between luma and chroma components. In addition to the Post Processing (PP) approach, the In-Loop Filtering (ILF) codec integration has also been considered, where the characteristics of the Group of Pictures (GoP) are taken into account to boost the performance. The proposed CNN-based Quality Enhancement (QE) framework has been implemented on top of the Versatile Video Coding (VVC) Test Model (VTM-10). Experiments show that the prediction-aware aspect of the proposed method improves the coding efficiency gain of the default CNN-based QE method by 1.52%, in terms of BD-BR, at the same network complexity compared to the default CNN-based QE filter.https://ieeexplore.ieee.org/document/9465693/CNNVVCquality enhancementin-loop filteringpost-processing |
| spellingShingle | Fatemeh Nasiri Wassim Hamidouche Luce Morin Nicolas Dhollande Gildas Cocherel A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC IEEE Open Journal of Signal Processing CNN VVC quality enhancement in-loop filtering post-processing |
| title | A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC |
| title_full | A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC |
| title_fullStr | A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC |
| title_full_unstemmed | A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC |
| title_short | A CNN-Based Prediction-Aware Quality Enhancement Framework for VVC |
| title_sort | cnn based prediction aware quality enhancement framework for vvc |
| topic | CNN VVC quality enhancement in-loop filtering post-processing |
| url | https://ieeexplore.ieee.org/document/9465693/ |
| work_keys_str_mv | AT fatemehnasiri acnnbasedpredictionawarequalityenhancementframeworkforvvc AT wassimhamidouche acnnbasedpredictionawarequalityenhancementframeworkforvvc AT lucemorin acnnbasedpredictionawarequalityenhancementframeworkforvvc AT nicolasdhollande acnnbasedpredictionawarequalityenhancementframeworkforvvc AT gildascocherel acnnbasedpredictionawarequalityenhancementframeworkforvvc AT fatemehnasiri cnnbasedpredictionawarequalityenhancementframeworkforvvc AT wassimhamidouche cnnbasedpredictionawarequalityenhancementframeworkforvvc AT lucemorin cnnbasedpredictionawarequalityenhancementframeworkforvvc AT nicolasdhollande cnnbasedpredictionawarequalityenhancementframeworkforvvc AT gildascocherel cnnbasedpredictionawarequalityenhancementframeworkforvvc |