On Relations Between the Relative Entropy and <em>χ</em><sup>2</sup>-Divergence, Generalizations and Applications

The relative entropy and the chi-squared divergence are fundamental divergence measures in information theory and statistics. This paper is focused on a study of integral relations between the two divergences, the implications of these relations, their information-theoretic applications, and some ge...

Full description

Bibliographic Details
Main Authors: Tomohiro Nishiyama, Igal Sason
Format: Article
Language:English
Published: MDPI AG 2020-05-01
Series:Entropy
Subjects:
Online Access:https://www.mdpi.com/1099-4300/22/5/563
Description
Summary:The relative entropy and the chi-squared divergence are fundamental divergence measures in information theory and statistics. This paper is focused on a study of integral relations between the two divergences, the implications of these relations, their information-theoretic applications, and some generalizations pertaining to the rich class of <em>f</em>-divergences. Applications that are studied in this paper refer to lossless compression, the method of types and large deviations, strong data–processing inequalities, bounds on contraction coefficients and maximal correlation, and the convergence rate to stationarity of a type of discrete-time Markov chains.
ISSN:1099-4300