Bitstream-corrupted JPEG images are restorable: two-stage compensation and alignment framework for image restoration
In this paper, we study a real-world JPEG image restoration problem with bit errors on the encrypted bitstream. The bit errors bring unpredictable color casts and block shifts on decoded image contents, which cannot be resolved by existing image restoration methods mainly relying on pre-defined degr...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Conference Paper |
| Language: | English |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/173559 |
| _version_ | 1824456516942430208 |
|---|---|
| author | Liu, Wenyang Wang, Yi Yap, Kim-Hui Chau, Lap-Pui |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Liu, Wenyang Wang, Yi Yap, Kim-Hui Chau, Lap-Pui |
| author_sort | Liu, Wenyang |
| collection | NTU |
| description | In this paper, we study a real-world JPEG image restoration problem with bit errors on the encrypted bitstream. The bit errors bring unpredictable color casts and block shifts on decoded image contents, which cannot be resolved by existing image restoration methods mainly relying on pre-defined degradation models in the pixel domain. To address these challenges, we propose a robust JPEG decoder, followed by a two-stage compensation and alignment framework to restore bitstream-corrupted JPEC images. Specifically, the robust JPEC decoder adopts an error-resilient mechanism to decode the corrupted JPEG bitstream. The two-stage framework is composed of the self-compensation and alignment (SCA) stage and the guided-compensation and alignment (GCA) stage. The SCA adaptively performs block-wise image color compensation and alignment based on the estimated color and block offsets via image content similarity. The GCA leverages the extracted low-resolution thumbnail from the JPEG header to guide full-resolution pixel-wise image restoration in a coarse-to-fine manner. It is achieved by a coarse-guided pix2pix network and a refine-guided bi-directional Laplacian pyramid fusion network. We conduct experiments on three benchmarks with varying degrees of bit error rates. Experimental results and ablation studies demonstrate the superiority of our proposed method. The code will be released at https://github.com/wenyang001/Two-ACIR. |
| first_indexed | 2025-02-19T03:55:21Z |
| format | Conference Paper |
| id | ntu-10356/173559 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2025-02-19T03:55:21Z |
| publishDate | 2024 |
| record_format | dspace |
| spelling | ntu-10356/1735592024-02-16T15:38:31Z Bitstream-corrupted JPEG images are restorable: two-stage compensation and alignment framework for image restoration Liu, Wenyang Wang, Yi Yap, Kim-Hui Chau, Lap-Pui School of Electrical and Electronic Engineering 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Engineering Low-level vision Image resolution In this paper, we study a real-world JPEG image restoration problem with bit errors on the encrypted bitstream. The bit errors bring unpredictable color casts and block shifts on decoded image contents, which cannot be resolved by existing image restoration methods mainly relying on pre-defined degradation models in the pixel domain. To address these challenges, we propose a robust JPEG decoder, followed by a two-stage compensation and alignment framework to restore bitstream-corrupted JPEC images. Specifically, the robust JPEC decoder adopts an error-resilient mechanism to decode the corrupted JPEG bitstream. The two-stage framework is composed of the self-compensation and alignment (SCA) stage and the guided-compensation and alignment (GCA) stage. The SCA adaptively performs block-wise image color compensation and alignment based on the estimated color and block offsets via image content similarity. The GCA leverages the extracted low-resolution thumbnail from the JPEG header to guide full-resolution pixel-wise image restoration in a coarse-to-fine manner. It is achieved by a coarse-guided pix2pix network and a refine-guided bi-directional Laplacian pyramid fusion network. We conduct experiments on three benchmarks with varying degrees of bit error rates. Experimental results and ablation studies demonstrate the superiority of our proposed method. The code will be released at https://github.com/wenyang001/Two-ACIR. National Research Foundation (NRF) Submitted/Accepted version This research/project is supported by the National Research Foundation, Singapore, and Cyber Security Agency of Singapore under its National Cybersecurity R&D Programme (NRF2018NCR-NCR009-0001). 2024-02-14T06:10:22Z 2024-02-14T06:10:22Z 2023 Conference Paper Liu, W., Wang, Y., Yap, K. & Chau, L. (2023). Bitstream-corrupted JPEG images are restorable: two-stage compensation and alignment framework for image restoration. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9979-9988. https://dx.doi.org/10.1109/CVPR52729.2023.00962 9798350301298 https://hdl.handle.net/10356/173559 10.1109/CVPR52729.2023.00962 2-s2.0-85173922230 9979 9988 en NRF2018 NCR-NCR009-0001 10.21979/N9/4CTXCI © 2023 IEEE. All rights reserved. This article may be downloaded for personal use only. Any other use requires prior permission of the copyright holder. The Version of Record is available online at http://doi.org/10.1109/CVPR52729.2023.00962. application/pdf |
| spellingShingle | Engineering Low-level vision Image resolution Liu, Wenyang Wang, Yi Yap, Kim-Hui Chau, Lap-Pui Bitstream-corrupted JPEG images are restorable: two-stage compensation and alignment framework for image restoration |
| title | Bitstream-corrupted JPEG images are restorable: two-stage compensation and alignment framework for image restoration |
| title_full | Bitstream-corrupted JPEG images are restorable: two-stage compensation and alignment framework for image restoration |
| title_fullStr | Bitstream-corrupted JPEG images are restorable: two-stage compensation and alignment framework for image restoration |
| title_full_unstemmed | Bitstream-corrupted JPEG images are restorable: two-stage compensation and alignment framework for image restoration |
| title_short | Bitstream-corrupted JPEG images are restorable: two-stage compensation and alignment framework for image restoration |
| title_sort | bitstream corrupted jpeg images are restorable two stage compensation and alignment framework for image restoration |
| topic | Engineering Low-level vision Image resolution |
| url | https://hdl.handle.net/10356/173559 |
| work_keys_str_mv | AT liuwenyang bitstreamcorruptedjpegimagesarerestorabletwostagecompensationandalignmentframeworkforimagerestoration AT wangyi bitstreamcorruptedjpegimagesarerestorabletwostagecompensationandalignmentframeworkforimagerestoration AT yapkimhui bitstreamcorruptedjpegimagesarerestorabletwostagecompensationandalignmentframeworkforimagerestoration AT chaulappui bitstreamcorruptedjpegimagesarerestorabletwostagecompensationandalignmentframeworkforimagerestoration |