An Integral Representation of the Logarithmic Function with Applications in Information Theory
We explore a well-known integral representation of the logarithmic function, and demonstrate its usefulness in obtaining compact, easily computable exact formulas for quantities that involve expectations and higher moments of the logarithm of a positive random variable (or the logarithm of a sum of...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-12-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/22/1/51 |
| _version_ | 1817992294491488256 |
|---|---|
| author | Neri Merhav Igal Sason |
| author_facet | Neri Merhav Igal Sason |
| author_sort | Neri Merhav |
| collection | DOAJ |
| description | We explore a well-known integral representation of the logarithmic function, and demonstrate its usefulness in obtaining compact, easily computable exact formulas for quantities that involve expectations and higher moments of the logarithm of a positive random variable (or the logarithm of a sum of i.i.d. positive random variables). The integral representation of the logarithm is proved useful in a variety of information-theoretic applications, including universal lossless data compression, entropy and differential entropy evaluations, and the calculation of the ergodic capacity of the single-input, multiple-output (SIMO) Gaussian channel with random parameters (known to both transmitter and receiver). This integral representation and its variants are anticipated to serve as a useful tool in additional applications, as a rigorous alternative to the popular (but non-rigorous) replica method (at least in some situations). |
| first_indexed | 2024-04-14T01:24:26Z |
| format | Article |
| id | doaj.art-7f77094e86c5473b8a2b149fb0b2463d |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-04-14T01:24:26Z |
| publishDate | 2019-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-7f77094e86c5473b8a2b149fb0b2463d2022-12-22T02:20:29ZengMDPI AGEntropy1099-43002019-12-012215110.3390/e22010051e22010051An Integral Representation of the Logarithmic Function with Applications in Information TheoryNeri Merhav0Igal Sason1The Andrew and Erna Viterbi Faculty of Electrical Engineering, Israel Institute of Technology Technion City, Haifa 3200003, IsraelThe Andrew and Erna Viterbi Faculty of Electrical Engineering, Israel Institute of Technology Technion City, Haifa 3200003, IsraelWe explore a well-known integral representation of the logarithmic function, and demonstrate its usefulness in obtaining compact, easily computable exact formulas for quantities that involve expectations and higher moments of the logarithm of a positive random variable (or the logarithm of a sum of i.i.d. positive random variables). The integral representation of the logarithm is proved useful in a variety of information-theoretic applications, including universal lossless data compression, entropy and differential entropy evaluations, and the calculation of the ergodic capacity of the single-input, multiple-output (SIMO) Gaussian channel with random parameters (known to both transmitter and receiver). This integral representation and its variants are anticipated to serve as a useful tool in additional applications, as a rigorous alternative to the popular (but non-rigorous) replica method (at least in some situations).https://www.mdpi.com/1099-4300/22/1/51integral representationlogarithmic expectationuniversal data compressionentropydifferential entropyergodic capacitysimo channelmultivariate cauchy distribution |
| spellingShingle | Neri Merhav Igal Sason An Integral Representation of the Logarithmic Function with Applications in Information Theory Entropy integral representation logarithmic expectation universal data compression entropy differential entropy ergodic capacity simo channel multivariate cauchy distribution |
| title | An Integral Representation of the Logarithmic Function with Applications in Information Theory |
| title_full | An Integral Representation of the Logarithmic Function with Applications in Information Theory |
| title_fullStr | An Integral Representation of the Logarithmic Function with Applications in Information Theory |
| title_full_unstemmed | An Integral Representation of the Logarithmic Function with Applications in Information Theory |
| title_short | An Integral Representation of the Logarithmic Function with Applications in Information Theory |
| title_sort | integral representation of the logarithmic function with applications in information theory |
| topic | integral representation logarithmic expectation universal data compression entropy differential entropy ergodic capacity simo channel multivariate cauchy distribution |
| url | https://www.mdpi.com/1099-4300/22/1/51 |
| work_keys_str_mv | AT nerimerhav anintegralrepresentationofthelogarithmicfunctionwithapplicationsininformationtheory AT igalsason anintegralrepresentationofthelogarithmicfunctionwithapplicationsininformationtheory AT nerimerhav integralrepresentationofthelogarithmicfunctionwithapplicationsininformationtheory AT igalsason integralrepresentationofthelogarithmicfunctionwithapplicationsininformationtheory |