Orders between Channels and Implications for Partial Information Decomposition
The partial information decomposition (PID) framework is concerned with decomposing the information that a set of random variables has with respect to a target variable into three types of components: redundant, synergistic, and unique. Classical information theory alone does not provide a unique wa...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/25/7/975 |
| _version_ | 1827733197708853248 |
|---|---|
| author | André F. C. Gomes Mário A. T. Figueiredo |
| author_facet | André F. C. Gomes Mário A. T. Figueiredo |
| author_sort | André F. C. Gomes |
| collection | DOAJ |
| description | The partial information decomposition (PID) framework is concerned with decomposing the information that a set of random variables has with respect to a target variable into three types of components: redundant, synergistic, and unique. Classical information theory alone does not provide a unique way to decompose information in this manner, and additional assumptions have to be made. Recently, Kolchinsky proposed a new general axiomatic approach to obtain measures of redundant information based on choosing an order relation between information sources (equivalently, order between communication channels). In this paper, we exploit this approach to introduce three new measures of redundant information (and the resulting decompositions) based on well-known preorders between channels, contributing to the enrichment of the PID landscape. We relate the new decompositions to existing ones, study several of their properties, and provide examples illustrating their novelty. As a side result, we prove that any preorder that satisfies Kolchinsky’s axioms yields a decomposition that meets the axioms originally introduced by Williams and Beer when they first proposed PID. |
| first_indexed | 2024-03-11T01:07:04Z |
| format | Article |
| id | doaj.art-c4729a15099e4ff0a553bdd899aa04e2 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-11T01:07:04Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-c4729a15099e4ff0a553bdd899aa04e22023-11-18T19:12:50ZengMDPI AGEntropy1099-43002023-06-0125797510.3390/e25070975Orders between Channels and Implications for Partial Information DecompositionAndré F. C. Gomes0Mário A. T. Figueiredo1Instituto de Telecomunicações and LUMLIS (Lisbon ELLIS Unit), Instituto Superior Técnico, Universidade de Lisboa, 1049-001 Lisboa, PortugalInstituto de Telecomunicações and LUMLIS (Lisbon ELLIS Unit), Instituto Superior Técnico, Universidade de Lisboa, 1049-001 Lisboa, PortugalThe partial information decomposition (PID) framework is concerned with decomposing the information that a set of random variables has with respect to a target variable into three types of components: redundant, synergistic, and unique. Classical information theory alone does not provide a unique way to decompose information in this manner, and additional assumptions have to be made. Recently, Kolchinsky proposed a new general axiomatic approach to obtain measures of redundant information based on choosing an order relation between information sources (equivalently, order between communication channels). In this paper, we exploit this approach to introduce three new measures of redundant information (and the resulting decompositions) based on well-known preorders between channels, contributing to the enrichment of the PID landscape. We relate the new decompositions to existing ones, study several of their properties, and provide examples illustrating their novelty. As a side result, we prove that any preorder that satisfies Kolchinsky’s axioms yields a decomposition that meets the axioms originally introduced by Williams and Beer when they first proposed PID.https://www.mdpi.com/1099-4300/25/7/975information theorypartial information decompositionchannel preordersintersection informationshared informationredundancy |
| spellingShingle | André F. C. Gomes Mário A. T. Figueiredo Orders between Channels and Implications for Partial Information Decomposition Entropy information theory partial information decomposition channel preorders intersection information shared information redundancy |
| title | Orders between Channels and Implications for Partial Information Decomposition |
| title_full | Orders between Channels and Implications for Partial Information Decomposition |
| title_fullStr | Orders between Channels and Implications for Partial Information Decomposition |
| title_full_unstemmed | Orders between Channels and Implications for Partial Information Decomposition |
| title_short | Orders between Channels and Implications for Partial Information Decomposition |
| title_sort | orders between channels and implications for partial information decomposition |
| topic | information theory partial information decomposition channel preorders intersection information shared information redundancy |
| url | https://www.mdpi.com/1099-4300/25/7/975 |
| work_keys_str_mv | AT andrefcgomes ordersbetweenchannelsandimplicationsforpartialinformationdecomposition AT marioatfigueiredo ordersbetweenchannelsandimplicationsforpartialinformationdecomposition |