Information-Distilling Quantizers
IEEE Let X and Y be dependent random variables. This paper considers the problem of designing a scalar quantizer for Y to maximize the mutual information between the quantizer’s output and X, and develops fundamental properties and bounds for this form of quantization, which is connected...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2022
|
| Online Access: | https://hdl.handle.net/1721.1/143839 |
| _version_ | 1826196767999262720 |
|---|---|
| author | Bhatt, Alankrita Nazer, Bobak Ordentlich, Or Polyanskiy, Yury |
| author2 | Massachusetts Institute of Technology. Laboratory for Information and Decision Systems |
| author_facet | Massachusetts Institute of Technology. Laboratory for Information and Decision Systems Bhatt, Alankrita Nazer, Bobak Ordentlich, Or Polyanskiy, Yury |
| author_sort | Bhatt, Alankrita |
| collection | MIT |
| description | IEEE Let X and Y be dependent random variables. This paper considers the problem of designing a scalar quantizer for Y to maximize the mutual information between the quantizer’s output and X, and develops fundamental properties and bounds for this form of quantization, which is connected to the log-loss distortion criterion. The main focus is the regime of low I(X; Y ), where it is shown that, if X is binary, a constant fraction of the mutual information can always be preserved using O(log(1/I(X; Y ))) quantization levels, and there exist distributions for which this many quantization levels are necessary. Furthermore, for larger finite alphabets 2 < |X| < ∞, it is established that an η-fraction of the mutual information can be preserved using roughly (log(|X|/I(X; Y )))η·(|X|-1) quantization levels. |
| first_indexed | 2024-09-23T10:37:42Z |
| format | Article |
| id | mit-1721.1/143839 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T10:37:42Z |
| publishDate | 2022 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1438392023-04-18T19:51:09Z Information-Distilling Quantizers Bhatt, Alankrita Nazer, Bobak Ordentlich, Or Polyanskiy, Yury Massachusetts Institute of Technology. Laboratory for Information and Decision Systems Statistics and Data Science Center (Massachusetts Institute of Technology) IEEE Let X and Y be dependent random variables. This paper considers the problem of designing a scalar quantizer for Y to maximize the mutual information between the quantizer’s output and X, and develops fundamental properties and bounds for this form of quantization, which is connected to the log-loss distortion criterion. The main focus is the regime of low I(X; Y ), where it is shown that, if X is binary, a constant fraction of the mutual information can always be preserved using O(log(1/I(X; Y ))) quantization levels, and there exist distributions for which this many quantization levels are necessary. Furthermore, for larger finite alphabets 2 < |X| < ∞, it is established that an η-fraction of the mutual information can be preserved using roughly (log(|X|/I(X; Y )))η·(|X|-1) quantization levels. 2022-07-19T12:10:37Z 2022-07-19T12:10:37Z 2021 2022-07-19T12:05:35Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/143839 Bhatt, Alankrita, Nazer, Bobak, Ordentlich, Or and Polyanskiy, Yury. 2021. "Information-Distilling Quantizers." IEEE Transactions on Information Theory, 67 (4). en 10.1109/TIT.2021.3059338 IEEE Transactions on Information Theory Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) MIT web domain |
| spellingShingle | Bhatt, Alankrita Nazer, Bobak Ordentlich, Or Polyanskiy, Yury Information-Distilling Quantizers |
| title | Information-Distilling Quantizers |
| title_full | Information-Distilling Quantizers |
| title_fullStr | Information-Distilling Quantizers |
| title_full_unstemmed | Information-Distilling Quantizers |
| title_short | Information-Distilling Quantizers |
| title_sort | information distilling quantizers |
| url | https://hdl.handle.net/1721.1/143839 |
| work_keys_str_mv | AT bhattalankrita informationdistillingquantizers AT nazerbobak informationdistillingquantizers AT ordentlichor informationdistillingquantizers AT polyanskiyyury informationdistillingquantizers |