Entropy Evaluation Based on Value Validity
Besides its importance in statistical physics and information theory, the Boltzmann-Shannon entropy S has become one of the most widely used and misused summary measures of various attributes (characteristics) in diverse fields of study. It has also been the subject of extensive and perhaps excessiv...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2014-09-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1099-4300/16/9/4855 |
| _version_ | 1811187056218472448 |
|---|---|
| author | Tarald O. Kvålseth |
| author_facet | Tarald O. Kvålseth |
| author_sort | Tarald O. Kvålseth |
| collection | DOAJ |
| description | Besides its importance in statistical physics and information theory, the Boltzmann-Shannon entropy S has become one of the most widely used and misused summary measures of various attributes (characteristics) in diverse fields of study. It has also been the subject of extensive and perhaps excessive generalizations. This paper introduces the concept and criteria for value validity as a means of determining if an entropy takes on values that reasonably reflect the attribute being measured and that permit different types of comparisons to be made for different probability distributions. While neither S nor its relative entropy equivalent S* meet the value-validity conditions, certain power functions of S and S* do to a considerable extent. No parametric generalization offers any advantage over S in this regard. A measure based on Euclidean distances between probability distributions is introduced as a potential entropy that does comply fully with the value-validity requirements and its statistical inference procedure is discussed. |
| first_indexed | 2024-04-11T13:55:40Z |
| format | Article |
| id | doaj.art-1dde766676fa4496a007c3a08da78916 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-04-11T13:55:40Z |
| publishDate | 2014-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-1dde766676fa4496a007c3a08da789162022-12-22T04:20:20ZengMDPI AGEntropy1099-43002014-09-011694855487310.3390/e16094855e16094855Entropy Evaluation Based on Value ValidityTarald O. Kvålseth0Department of Mechanical Engineering, University of Minnesota, Minneapolis, MN 55455, USABesides its importance in statistical physics and information theory, the Boltzmann-Shannon entropy S has become one of the most widely used and misused summary measures of various attributes (characteristics) in diverse fields of study. It has also been the subject of extensive and perhaps excessive generalizations. This paper introduces the concept and criteria for value validity as a means of determining if an entropy takes on values that reasonably reflect the attribute being measured and that permit different types of comparisons to be made for different probability distributions. While neither S nor its relative entropy equivalent S* meet the value-validity conditions, certain power functions of S and S* do to a considerable extent. No parametric generalization offers any advantage over S in this regard. A measure based on Euclidean distances between probability distributions is introduced as a potential entropy that does comply fully with the value-validity requirements and its statistical inference procedure is discussed.http://www.mdpi.com/1099-4300/16/9/4855entropyrelative entropygeneralized entropiesentropy validityEuclidean entropy |
| spellingShingle | Tarald O. Kvålseth Entropy Evaluation Based on Value Validity Entropy entropy relative entropy generalized entropies entropy validity Euclidean entropy |
| title | Entropy Evaluation Based on Value Validity |
| title_full | Entropy Evaluation Based on Value Validity |
| title_fullStr | Entropy Evaluation Based on Value Validity |
| title_full_unstemmed | Entropy Evaluation Based on Value Validity |
| title_short | Entropy Evaluation Based on Value Validity |
| title_sort | entropy evaluation based on value validity |
| topic | entropy relative entropy generalized entropies entropy validity Euclidean entropy |
| url | http://www.mdpi.com/1099-4300/16/9/4855 |
| work_keys_str_mv | AT taraldokvalseth entropyevaluationbasedonvaluevalidity |