Empirical Estimation of Information Measures: A Literature Guide
We give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasi...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-07-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/21/8/720 |
| _version_ | 1811187333852037120 |
|---|---|
| author | Sergio Verdú |
| author_facet | Sergio Verdú |
| author_sort | Sergio Verdú |
| collection | DOAJ |
| description | We give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasingly important in data science, machine learning, biology, neuroscience, economics, language, and other experimental sciences. |
| first_indexed | 2024-04-11T14:01:30Z |
| format | Article |
| id | doaj.art-68415e81d85b401ea971425476b4550a |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-04-11T14:01:30Z |
| publishDate | 2019-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-68415e81d85b401ea971425476b4550a2022-12-22T04:20:07ZengMDPI AGEntropy1099-43002019-07-0121872010.3390/e21080720e21080720Empirical Estimation of Information Measures: A Literature GuideSergio Verdú0Independent Researcher, Princeton, NJ 08540, USAWe give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasingly important in data science, machine learning, biology, neuroscience, economics, language, and other experimental sciences.https://www.mdpi.com/1099-4300/21/8/720information measuresempirical estimatorsentropyrelative entropymutual informationuniversal estimation |
| spellingShingle | Sergio Verdú Empirical Estimation of Information Measures: A Literature Guide Entropy information measures empirical estimators entropy relative entropy mutual information universal estimation |
| title | Empirical Estimation of Information Measures: A Literature Guide |
| title_full | Empirical Estimation of Information Measures: A Literature Guide |
| title_fullStr | Empirical Estimation of Information Measures: A Literature Guide |
| title_full_unstemmed | Empirical Estimation of Information Measures: A Literature Guide |
| title_short | Empirical Estimation of Information Measures: A Literature Guide |
| title_sort | empirical estimation of information measures a literature guide |
| topic | information measures empirical estimators entropy relative entropy mutual information universal estimation |
| url | https://www.mdpi.com/1099-4300/21/8/720 |
| work_keys_str_mv | AT sergioverdu empiricalestimationofinformationmeasuresaliteratureguide |