| Summary: | Deep image coding (DIC) for hybrid application contexts has recently attracted significant research interest because of its potential to support both human and machine visual tasks. Since the regions of interest (ROI) are different for different application contexts, it is important to design an adaptive image coding mechanism in practical DIC. In this paper, we propose the first quantization-based adaptive DIC framework for hybrid contexts of image reconstruction and classification. This framework can be applied to upgrade existing fixed-rate DIC models into adaptive DIC for hybrid contexts. It consists of two key modules: a semantics-based ROI mask generation module and a module for generating ROI gain and inverse gain matrices. These matrices are used to control the quantization accuracy of different latent vector elements, thereby achieving encoding at different rates while prioritizing the reconstruction quality of the ROI. Moreover, we propose a five-stage training method for the quantization-based adaptive DIC model to optimize the rate-distortion-classification-perception (RDCP) tradeoff. Experiments over a wide rate range show that our method achieves superior RDCP tradeoff performance. Compared to the benchmark scheme BM-CHENG, the proposed algorithm improves the classification accuracy by an average of 15%. The average relative improvements on various metrics, such as natural image quality evaluator (NIQE), learned perceptual image patch similarity (LPIPS), and feature similarity index measure (FSIM), are about 22%, 47%, and 1%, respectively. The proposed algorithm is a promising candidate for fast adaptive coding with low-complexity constraints.
|
|---|