A Novel Approach to the Partial Information Decomposition
We consider the “partial information decomposition” (PID) problem, which aims to decompose the information that a set of source random variables provide about a target random variable into separate redundant, synergistic, union, and unique components. In the first part of this paper, we propose a ge...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/24/3/403 |
| _version_ | 1797446651950923776 |
|---|---|
| author | Artemy Kolchinsky |
| author_facet | Artemy Kolchinsky |
| author_sort | Artemy Kolchinsky |
| collection | DOAJ |
| description | We consider the “partial information decomposition” (PID) problem, which aims to decompose the information that a set of source random variables provide about a target random variable into separate redundant, synergistic, union, and unique components. In the first part of this paper, we propose a general framework for constructing a multivariate PID. Our framework is defined in terms of a formal analogy with intersection and union from set theory, along with an ordering relation which specifies when one information source is more informative than another. Our definitions are algebraically and axiomatically motivated, and can be generalized to domains beyond Shannon information theory (such as algorithmic information theory and quantum information theory). In the second part of this paper, we use our general framework to define a PID in terms of the well-known Blackwell order, which has a fundamental operational interpretation. We demonstrate our approach on numerous examples and show that it overcomes many drawbacks associated with previous proposals. |
| first_indexed | 2024-03-09T13:44:35Z |
| format | Article |
| id | doaj.art-c70b9a6bd64a4a5bbf32bfb57bc231e7 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-09T13:44:35Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-c70b9a6bd64a4a5bbf32bfb57bc231e72023-11-30T21:03:09ZengMDPI AGEntropy1099-43002022-03-0124340310.3390/e24030403A Novel Approach to the Partial Information DecompositionArtemy Kolchinsky0Santa Fe Institute, Santa Fe, NM 87501, USAWe consider the “partial information decomposition” (PID) problem, which aims to decompose the information that a set of source random variables provide about a target random variable into separate redundant, synergistic, union, and unique components. In the first part of this paper, we propose a general framework for constructing a multivariate PID. Our framework is defined in terms of a formal analogy with intersection and union from set theory, along with an ordering relation which specifies when one information source is more informative than another. Our definitions are algebraically and axiomatically motivated, and can be generalized to domains beyond Shannon information theory (such as algorithmic information theory and quantum information theory). In the second part of this paper, we use our general framework to define a PID in terms of the well-known Blackwell order, which has a fundamental operational interpretation. We demonstrate our approach on numerous examples and show that it overcomes many drawbacks associated with previous proposals.https://www.mdpi.com/1099-4300/24/3/403partial information decompositionredundancysynergy |
| spellingShingle | Artemy Kolchinsky A Novel Approach to the Partial Information Decomposition Entropy partial information decomposition redundancy synergy |
| title | A Novel Approach to the Partial Information Decomposition |
| title_full | A Novel Approach to the Partial Information Decomposition |
| title_fullStr | A Novel Approach to the Partial Information Decomposition |
| title_full_unstemmed | A Novel Approach to the Partial Information Decomposition |
| title_short | A Novel Approach to the Partial Information Decomposition |
| title_sort | novel approach to the partial information decomposition |
| topic | partial information decomposition redundancy synergy |
| url | https://www.mdpi.com/1099-4300/24/3/403 |
| work_keys_str_mv | AT artemykolchinsky anovelapproachtothepartialinformationdecomposition AT artemykolchinsky novelapproachtothepartialinformationdecomposition |