Bounds between contraction coefficients
In this paper, we delineate how the contraction coefficient of the strong data processing inequality for KL divergence can be used to learn likelihood models. We then present an alternative formulation that forces the input KL divergence to vanish, and achieves a contraction coefficient equivalent t...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2017
|
| Online Access: | http://hdl.handle.net/1721.1/112984 https://orcid.org/0000-0002-2978-8116 https://orcid.org/0000-0002-6108-0222 |
| _version_ | 1811096080533684224 |
|---|---|
| author | Makur, Anuran Zheng, Lizhong |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Makur, Anuran Zheng, Lizhong |
| author_sort | Makur, Anuran |
| collection | MIT |
| description | In this paper, we delineate how the contraction coefficient of the strong data processing inequality for KL divergence can be used to learn likelihood models. We then present an alternative formulation that forces the input KL divergence to vanish, and achieves a contraction coefficient equivalent to the squared maximal correlation using a linear algebraic solution. To analyze the performance loss in using this simple but suboptimal procedure, we bound these coefficients in the discrete and finite regime, and prove their equivalence in the Gaussian regime. |
| first_indexed | 2024-09-23T16:38:02Z |
| format | Article |
| id | mit-1721.1/112984 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T16:38:02Z |
| publishDate | 2017 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1129842022-10-02T08:36:36Z Bounds between contraction coefficients Makur, Anuran Zheng, Lizhong Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Makur, Anuran Zheng, Lizhong In this paper, we delineate how the contraction coefficient of the strong data processing inequality for KL divergence can be used to learn likelihood models. We then present an alternative formulation that forces the input KL divergence to vanish, and achieves a contraction coefficient equivalent to the squared maximal correlation using a linear algebraic solution. To analyze the performance loss in using this simple but suboptimal procedure, we bound these coefficients in the discrete and finite regime, and prove their equivalence in the Gaussian regime. 2017-12-29T19:01:06Z 2017-12-29T19:01:06Z 2016-04 2015-09 Article http://purl.org/eprint/type/ConferencePaper 978-1-5090-1824-6 http://hdl.handle.net/1721.1/112984 Makur, Anuran, and Lizhong Zheng. "Bounds between Contraction Coefficients." 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), 29 September - 2 October, 2015, Monticello, IL, IEEE, 2015, pp. 1422–29. https://orcid.org/0000-0002-2978-8116 https://orcid.org/0000-0002-6108-0222 en_US http://dx.doi.org/10.1109/ALLERTON.2015.7447175 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) arXiv |
| spellingShingle | Makur, Anuran Zheng, Lizhong Bounds between contraction coefficients |
| title | Bounds between contraction coefficients |
| title_full | Bounds between contraction coefficients |
| title_fullStr | Bounds between contraction coefficients |
| title_full_unstemmed | Bounds between contraction coefficients |
| title_short | Bounds between contraction coefficients |
| title_sort | bounds between contraction coefficients |
| url | http://hdl.handle.net/1721.1/112984 https://orcid.org/0000-0002-2978-8116 https://orcid.org/0000-0002-6108-0222 |
| work_keys_str_mv | AT makuranuran boundsbetweencontractioncoefficients AT zhenglizhong boundsbetweencontractioncoefficients |