On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information Radius
We generalize the Jensen-Shannon divergence and the Jensen-Shannon diversity index by considering a variational definition with respect to a generic mean, thereby extending the notion of Sibson’s information radius. The variational definition applies to any arbitrary distance and yields a new way to...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-04-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/23/4/464 |
| _version_ | 1797537646470234112 |
|---|---|
| author | Frank Nielsen |
| author_facet | Frank Nielsen |
| author_sort | Frank Nielsen |
| collection | DOAJ |
| description | We generalize the Jensen-Shannon divergence and the Jensen-Shannon diversity index by considering a variational definition with respect to a generic mean, thereby extending the notion of Sibson’s information radius. The variational definition applies to any arbitrary distance and yields a new way to define a Jensen-Shannon symmetrization of distances. When the variational optimization is further constrained to belong to prescribed families of probability measures, we get relative Jensen-Shannon divergences and their equivalent Jensen-Shannon symmetrizations of distances that generalize the concept of information projections. Finally, we touch upon applications of these variational Jensen-Shannon divergences and diversity indices to clustering and quantization tasks of probability measures, including statistical mixtures. |
| first_indexed | 2024-03-10T12:19:22Z |
| format | Article |
| id | doaj.art-44b27112aee84b90991f00ae959c2480 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T12:19:22Z |
| publishDate | 2021-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-44b27112aee84b90991f00ae959c24802023-11-21T15:36:12ZengMDPI AGEntropy1099-43002021-04-0123446410.3390/e23040464On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information RadiusFrank Nielsen0Sony Computer Science Laboratories, Tokyo 141-0022, JapanWe generalize the Jensen-Shannon divergence and the Jensen-Shannon diversity index by considering a variational definition with respect to a generic mean, thereby extending the notion of Sibson’s information radius. The variational definition applies to any arbitrary distance and yields a new way to define a Jensen-Shannon symmetrization of distances. When the variational optimization is further constrained to belong to prescribed families of probability measures, we get relative Jensen-Shannon divergences and their equivalent Jensen-Shannon symmetrizations of distances that generalize the concept of information projections. Finally, we touch upon applications of these variational Jensen-Shannon divergences and diversity indices to clustering and quantization tasks of probability measures, including statistical mixtures.https://www.mdpi.com/1099-4300/23/4/464Jensen-Shannon divergencediversity indexRényi entropyinformation radiusinformation projectionexponential family |
| spellingShingle | Frank Nielsen On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information Radius Entropy Jensen-Shannon divergence diversity index Rényi entropy information radius information projection exponential family |
| title | On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information Radius |
| title_full | On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information Radius |
| title_fullStr | On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information Radius |
| title_full_unstemmed | On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information Radius |
| title_short | On a Variational Definition for the Jensen-Shannon Symmetrization of Distances Based on the Information Radius |
| title_sort | on a variational definition for the jensen shannon symmetrization of distances based on the information radius |
| topic | Jensen-Shannon divergence diversity index Rényi entropy information radius information projection exponential family |
| url | https://www.mdpi.com/1099-4300/23/4/464 |
| work_keys_str_mv | AT franknielsen onavariationaldefinitionforthejensenshannonsymmetrizationofdistancesbasedontheinformationradius |