Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical Manifolds
Divergence functions are the non-symmetric “distance” on the manifold, Μθ, of parametric probability density functions over a measure space, (Χ,μ). Classical information geometry prescribes, on Μθ: (i) a Riemannian metric given by the Fisher information; (ii) a pair of dual connections (giving rise...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2013-12-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1099-4300/15/12/5384 |
| _version_ | 1811304200715370496 |
|---|---|
| author | Jun Zhang |
| author_facet | Jun Zhang |
| author_sort | Jun Zhang |
| collection | DOAJ |
| description | Divergence functions are the non-symmetric “distance” on the manifold, Μθ, of parametric probability density functions over a measure space, (Χ,μ). Classical information geometry prescribes, on Μθ: (i) a Riemannian metric given by the Fisher information; (ii) a pair of dual connections (giving rise to the family of α-connections) that preserve the metric under parallel transport by their joint actions; and (iii) a family of divergence functions ( α-divergence) defined on Μθ x Μθ, which induce the metric and the dual connections. Here, we construct an extension of this differential geometric structure from Μθ (that of parametric probability density functions) to the manifold, Μ, of non-parametric functions on X, removing the positivity and normalization constraints. The generalized Fisher information and α-connections on M are induced by an α-parameterized family of divergence functions, reflecting the fundamental convex inequality associated with any smooth and strictly convex function. The infinite-dimensional manifold, M, has zero curvature for all these α-connections; hence, the generally non-zero curvature of M can be interpreted as arising from an embedding of Μθ into Μ. Furthermore, when a parametric model (after a monotonic scaling) forms an affine submanifold, its natural and expectation parameters form biorthogonal coordinates, and such a submanifold is dually flat for α = ± 1, generalizing the results of Amari’s α-embedding. The present analysis illuminates two different types of duality in information geometry, one concerning the referential status of a point (measurable function) expressed in the divergence function (“referential duality”) and the other concerning its representation under an arbitrary monotone scaling (“representational duality”). |
| first_indexed | 2024-04-13T08:02:51Z |
| format | Article |
| id | doaj.art-dbfca6769c0144fb9861a0aa51e0bd6b |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-04-13T08:02:51Z |
| publishDate | 2013-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-dbfca6769c0144fb9861a0aa51e0bd6b2022-12-22T02:55:15ZengMDPI AGEntropy1099-43002013-12-0115125384541810.3390/e15125384e15125384Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical ManifoldsJun Zhang0Department of Psychology and Department of Mathematics, University of Michigan, 530 ChurchStreet, Ann Arbor, MI 48109, USADivergence functions are the non-symmetric “distance” on the manifold, Μθ, of parametric probability density functions over a measure space, (Χ,μ). Classical information geometry prescribes, on Μθ: (i) a Riemannian metric given by the Fisher information; (ii) a pair of dual connections (giving rise to the family of α-connections) that preserve the metric under parallel transport by their joint actions; and (iii) a family of divergence functions ( α-divergence) defined on Μθ x Μθ, which induce the metric and the dual connections. Here, we construct an extension of this differential geometric structure from Μθ (that of parametric probability density functions) to the manifold, Μ, of non-parametric functions on X, removing the positivity and normalization constraints. The generalized Fisher information and α-connections on M are induced by an α-parameterized family of divergence functions, reflecting the fundamental convex inequality associated with any smooth and strictly convex function. The infinite-dimensional manifold, M, has zero curvature for all these α-connections; hence, the generally non-zero curvature of M can be interpreted as arising from an embedding of Μθ into Μ. Furthermore, when a parametric model (after a monotonic scaling) forms an affine submanifold, its natural and expectation parameters form biorthogonal coordinates, and such a submanifold is dually flat for α = ± 1, generalizing the results of Amari’s α-embedding. The present analysis illuminates two different types of duality in information geometry, one concerning the referential status of a point (measurable function) expressed in the divergence function (“referential duality”) and the other concerning its representation under an arbitrary monotone scaling (“representational duality”).http://www.mdpi.com/1099-4300/15/12/5384Fisher informationalpha-connectioninfinite-dimensional manifoldconvex function |
| spellingShingle | Jun Zhang Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical Manifolds Entropy Fisher information alpha-connection infinite-dimensional manifold convex function |
| title | Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical Manifolds |
| title_full | Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical Manifolds |
| title_fullStr | Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical Manifolds |
| title_full_unstemmed | Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical Manifolds |
| title_short | Nonparametric Information Geometry: From Divergence Function to Referential-Representational Biduality on Statistical Manifolds |
| title_sort | nonparametric information geometry from divergence function to referential representational biduality on statistical manifolds |
| topic | Fisher information alpha-connection infinite-dimensional manifold convex function |
| url | http://www.mdpi.com/1099-4300/15/12/5384 |
| work_keys_str_mv | AT junzhang nonparametricinformationgeometryfromdivergencefunctiontoreferentialrepresentationalbidualityonstatisticalmanifolds |