Lenslet Image Coding With SAIs Synthesis via 3D CNNs-Based Reinforcement Learning With a Rate Reward
The deep learning-based coding schemes for lenslet images combine coding standards and view synthesis through Deep Learning (DL) models, where the compression efficiency is heavily influenced by the coding structure and quality of synthesized views. To exploit the inter-view redundancy among Sub-Ape...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10151925/ |
| _version_ | 1797794173100752896 |
|---|---|
| author | Xiaoda Zhong Tao Lu Diyang Xiao Rui Zhong |
| author_facet | Xiaoda Zhong Tao Lu Diyang Xiao Rui Zhong |
| author_sort | Xiaoda Zhong |
| collection | DOAJ |
| description | The deep learning-based coding schemes for lenslet images combine coding standards and view synthesis through Deep Learning (DL) models, where the compression efficiency is heavily influenced by the coding structure and quality of synthesized views. To exploit the inter-view redundancy among Sub-Aperture Images (SAIs), this paper proposes a hybrid closed-loop coding system that uses a novel coding structure based on checkerboard interleaving at a frame level. The frame-wise checkerboard interleaving method partitions an Original SAIs’ Set (OSS) of images into two mutually exclusive subsets, each consisting of alternating rows and columns of SAIs. We utilize the video coding standard Versatile Video Coding (VVC) to encode one subset while proposing a novel rate constraint-reinforced 3D Convolutional Neural Networks (CNNs) to predict the other subset, referred to as the complement subset. The rate constraint-reinforced 3D CNNs is newly designed with a gradient loss and reinforced rate cost to improve synthesized SAIs’ image quality and bit cost saving simultaneously. Experimental results on the light field image dataset demonstrate that the proposed hybrid coding system outperforms both HEVC_LDP and the previous state-of-the-art (SOTA), achieving an average BD-Bitrate savings of 41.58% and 23.31%, respectively. |
| first_indexed | 2024-03-13T02:57:57Z |
| format | Article |
| id | doaj.art-c1c47d88044544a08383291f789354d2 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-13T02:57:57Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c1c47d88044544a08383291f789354d22023-06-27T23:00:30ZengIEEEIEEE Access2169-35362023-01-0111624626247110.1109/ACCESS.2023.328629810151925Lenslet Image Coding With SAIs Synthesis via 3D CNNs-Based Reinforcement Learning With a Rate RewardXiaoda Zhong0https://orcid.org/0000-0002-4122-0670Tao Lu1Diyang Xiao2https://orcid.org/0009-0006-8650-4497Rui Zhong3Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, ChinaHubei Key Laboratory of Intelligent Robot, Wuhan Institute of Technology, Wuhan, ChinaSchool of Computer Science, Central China Normal University, Wuhan, ChinaSchool of Computer Science, Central China Normal University, Wuhan, ChinaThe deep learning-based coding schemes for lenslet images combine coding standards and view synthesis through Deep Learning (DL) models, where the compression efficiency is heavily influenced by the coding structure and quality of synthesized views. To exploit the inter-view redundancy among Sub-Aperture Images (SAIs), this paper proposes a hybrid closed-loop coding system that uses a novel coding structure based on checkerboard interleaving at a frame level. The frame-wise checkerboard interleaving method partitions an Original SAIs’ Set (OSS) of images into two mutually exclusive subsets, each consisting of alternating rows and columns of SAIs. We utilize the video coding standard Versatile Video Coding (VVC) to encode one subset while proposing a novel rate constraint-reinforced 3D Convolutional Neural Networks (CNNs) to predict the other subset, referred to as the complement subset. The rate constraint-reinforced 3D CNNs is newly designed with a gradient loss and reinforced rate cost to improve synthesized SAIs’ image quality and bit cost saving simultaneously. Experimental results on the light field image dataset demonstrate that the proposed hybrid coding system outperforms both HEVC_LDP and the previous state-of-the-art (SOTA), achieving an average BD-Bitrate savings of 41.58% and 23.31%, respectively.https://ieeexplore.ieee.org/document/10151925/Lenslet imagecompressionreinforcement learning3D CNNsVVC |
| spellingShingle | Xiaoda Zhong Tao Lu Diyang Xiao Rui Zhong Lenslet Image Coding With SAIs Synthesis via 3D CNNs-Based Reinforcement Learning With a Rate Reward IEEE Access Lenslet image compression reinforcement learning 3D CNNs VVC |
| title | Lenslet Image Coding With SAIs Synthesis via 3D CNNs-Based Reinforcement Learning With a Rate Reward |
| title_full | Lenslet Image Coding With SAIs Synthesis via 3D CNNs-Based Reinforcement Learning With a Rate Reward |
| title_fullStr | Lenslet Image Coding With SAIs Synthesis via 3D CNNs-Based Reinforcement Learning With a Rate Reward |
| title_full_unstemmed | Lenslet Image Coding With SAIs Synthesis via 3D CNNs-Based Reinforcement Learning With a Rate Reward |
| title_short | Lenslet Image Coding With SAIs Synthesis via 3D CNNs-Based Reinforcement Learning With a Rate Reward |
| title_sort | lenslet image coding with sais synthesis via 3d cnns based reinforcement learning with a rate reward |
| topic | Lenslet image compression reinforcement learning 3D CNNs VVC |
| url | https://ieeexplore.ieee.org/document/10151925/ |
| work_keys_str_mv | AT xiaodazhong lensletimagecodingwithsaissynthesisvia3dcnnsbasedreinforcementlearningwitharatereward AT taolu lensletimagecodingwithsaissynthesisvia3dcnnsbasedreinforcementlearningwitharatereward AT diyangxiao lensletimagecodingwithsaissynthesisvia3dcnnsbasedreinforcementlearningwitharatereward AT ruizhong lensletimagecodingwithsaissynthesisvia3dcnnsbasedreinforcementlearningwitharatereward |