Insights into Entropy as a Measure of Multivariate Variability
Entropy has been widely employed as a measure of variability for problems, such as machine learning and signal processing. In this paper, we provide some new insights into the behaviors of entropy as a measure of multivariate variability. The relationships between multivariate entropy (joint or tota...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2016-05-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1099-4300/18/5/196 |
| _version_ | 1811279985448583168 |
|---|---|
| author | Badong Chen Jianji Wang Haiquan Zhao Jose C. Principe |
| author_facet | Badong Chen Jianji Wang Haiquan Zhao Jose C. Principe |
| author_sort | Badong Chen |
| collection | DOAJ |
| description | Entropy has been widely employed as a measure of variability for problems, such as machine learning and signal processing. In this paper, we provide some new insights into the behaviors of entropy as a measure of multivariate variability. The relationships between multivariate entropy (joint or total marginal) and traditional measures of multivariate variability, such as total dispersion and generalized variance, are investigated. It is shown that for the jointly Gaussian case, the joint entropy (or entropy power) is equivalent to the generalized variance, while total marginal entropy is equivalent to the geometric mean of the marginal variances and total marginal entropy power is equivalent to the total dispersion. The smoothed multivariate entropy (joint or total marginal) and the kernel density estimation (KDE)-based entropy estimator (with finite samples) are also studied, which, under certain conditions, will be approximately equivalent to the total dispersion (or a total dispersion estimator), regardless of the data distribution. |
| first_indexed | 2024-04-13T01:05:36Z |
| format | Article |
| id | doaj.art-7af3793603c84efd9817404621a944a7 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-04-13T01:05:36Z |
| publishDate | 2016-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-7af3793603c84efd9817404621a944a72022-12-22T03:09:21ZengMDPI AGEntropy1099-43002016-05-0118519610.3390/e18050196e18050196Insights into Entropy as a Measure of Multivariate VariabilityBadong Chen0Jianji Wang1Haiquan Zhao2Jose C. Principe3School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, ChinaSchool of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, ChinaSchool of Electrical Engineering, Southwest Jiaotong University, Chengdu 610031, ChinaSchool of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, ChinaEntropy has been widely employed as a measure of variability for problems, such as machine learning and signal processing. In this paper, we provide some new insights into the behaviors of entropy as a measure of multivariate variability. The relationships between multivariate entropy (joint or total marginal) and traditional measures of multivariate variability, such as total dispersion and generalized variance, are investigated. It is shown that for the jointly Gaussian case, the joint entropy (or entropy power) is equivalent to the generalized variance, while total marginal entropy is equivalent to the geometric mean of the marginal variances and total marginal entropy power is equivalent to the total dispersion. The smoothed multivariate entropy (joint or total marginal) and the kernel density estimation (KDE)-based entropy estimator (with finite samples) are also studied, which, under certain conditions, will be approximately equivalent to the total dispersion (or a total dispersion estimator), regardless of the data distribution.http://www.mdpi.com/1099-4300/18/5/196entropysmoothed entropymultivariate variabilitygeneralized variancetotal dispersion |
| spellingShingle | Badong Chen Jianji Wang Haiquan Zhao Jose C. Principe Insights into Entropy as a Measure of Multivariate Variability Entropy entropy smoothed entropy multivariate variability generalized variance total dispersion |
| title | Insights into Entropy as a Measure of Multivariate Variability |
| title_full | Insights into Entropy as a Measure of Multivariate Variability |
| title_fullStr | Insights into Entropy as a Measure of Multivariate Variability |
| title_full_unstemmed | Insights into Entropy as a Measure of Multivariate Variability |
| title_short | Insights into Entropy as a Measure of Multivariate Variability |
| title_sort | insights into entropy as a measure of multivariate variability |
| topic | entropy smoothed entropy multivariate variability generalized variance total dispersion |
| url | http://www.mdpi.com/1099-4300/18/5/196 |
| work_keys_str_mv | AT badongchen insightsintoentropyasameasureofmultivariatevariability AT jianjiwang insightsintoentropyasameasureofmultivariatevariability AT haiquanzhao insightsintoentropyasameasureofmultivariatevariability AT josecprincipe insightsintoentropyasameasureofmultivariatevariability |