Quantifying Unique Information
We propose new measures of shared information, unique information and synergistic information that can be used to decompose the mutual information of a pair of random variables (Y, Z) with a third random variable X. Our measures are motivated by an operational idea of unique information, which sugge...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2014-04-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1099-4300/16/4/2161 |
| _version_ | 1811306716589981696 |
|---|---|
| author | Nils Bertschinger Johannes Rauh Eckehard Olbrich Jürgen Jost Nihat Ay |
| author_facet | Nils Bertschinger Johannes Rauh Eckehard Olbrich Jürgen Jost Nihat Ay |
| author_sort | Nils Bertschinger |
| collection | DOAJ |
| description | We propose new measures of shared information, unique information and synergistic information that can be used to decompose the mutual information of a pair of random variables (Y, Z) with a third random variable X. Our measures are motivated by an operational idea of unique information, which suggests that shared information and unique information should depend only on the marginal distributions of the pairs (X, Y) and (X,Z). Although this invariance property has not been studied before, it is satisfied by other proposed measures of shared information. The invariance property does not uniquely determine our new measures, but it implies that the functions that we define are bounds to any other measures satisfying the same invariance property. We study properties of our measures and compare them to other candidate measures. |
| first_indexed | 2024-04-13T08:50:28Z |
| format | Article |
| id | doaj.art-e95b48f2bba242cdb4f6d4c328df0ab0 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-04-13T08:50:28Z |
| publishDate | 2014-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-e95b48f2bba242cdb4f6d4c328df0ab02022-12-22T02:53:31ZengMDPI AGEntropy1099-43002014-04-011642161218310.3390/e16042161e16042161Quantifying Unique InformationNils Bertschinger0Johannes Rauh1Eckehard Olbrich2Jürgen Jost3Nihat Ay4Max Planck Institute for Mathematics in the Sciences, Inselstraße 23, 04109 Leipzig, GermanyMax Planck Institute for Mathematics in the Sciences, Inselstraße 23, 04109 Leipzig, GermanyMax Planck Institute for Mathematics in the Sciences, Inselstraße 23, 04109 Leipzig, GermanyMax Planck Institute for Mathematics in the Sciences, Inselstraße 23, 04109 Leipzig, GermanyMax Planck Institute for Mathematics in the Sciences, Inselstraße 23, 04109 Leipzig, GermanyWe propose new measures of shared information, unique information and synergistic information that can be used to decompose the mutual information of a pair of random variables (Y, Z) with a third random variable X. Our measures are motivated by an operational idea of unique information, which suggests that shared information and unique information should depend only on the marginal distributions of the pairs (X, Y) and (X,Z). Although this invariance property has not been studied before, it is satisfied by other proposed measures of shared information. The invariance property does not uniquely determine our new measures, but it implies that the functions that we define are bounds to any other measures satisfying the same invariance property. We study properties of our measures and compare them to other candidate measures.http://www.mdpi.com/1099-4300/16/4/2161Shannon informationmutual informationinformation decompositionshared informationsynergy |
| spellingShingle | Nils Bertschinger Johannes Rauh Eckehard Olbrich Jürgen Jost Nihat Ay Quantifying Unique Information Entropy Shannon information mutual information information decomposition shared information synergy |
| title | Quantifying Unique Information |
| title_full | Quantifying Unique Information |
| title_fullStr | Quantifying Unique Information |
| title_full_unstemmed | Quantifying Unique Information |
| title_short | Quantifying Unique Information |
| title_sort | quantifying unique information |
| topic | Shannon information mutual information information decomposition shared information synergy |
| url | http://www.mdpi.com/1099-4300/16/4/2161 |
| work_keys_str_mv | AT nilsbertschinger quantifyinguniqueinformation AT johannesrauh quantifyinguniqueinformation AT eckehardolbrich quantifyinguniqueinformation AT jurgenjost quantifyinguniqueinformation AT nihatay quantifyinguniqueinformation |