Hierarchical Quantification of Synergy in Channels
The decomposition of channel information into synergies of different order is an open, active problem in the theory of complex systems. Most approaches to the problem are based on information theory, and propose decompositions of mutual information between inputs and outputs in se-veral ways, none o...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2016-01-01
|
| Series: | Frontiers in Robotics and AI |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/Journal/10.3389/frobt.2015.00035/full |
| _version_ | 1828295660521127936 |
|---|---|
| author | Paolo ePerrone Nihat eAy |
| author_facet | Paolo ePerrone Nihat eAy |
| author_sort | Paolo ePerrone |
| collection | DOAJ |
| description | The decomposition of channel information into synergies of different order is an open, active problem in the theory of complex systems. Most approaches to the problem are based on information theory, and propose decompositions of mutual information between inputs and outputs in se-veral ways, none of which is generally accepted yet.We propose a new point of view on the topic. We model a multi-input channel as a Markov kernel. We can project the channel onto a series of exponential families which form a hierarchical structure. This is carried out with tools from information geometry, in a way analogous to the projections of probability distributions introduced by Amari. A Pythagorean relation leads naturally to a decomposition of the mutual information between inputs and outputs into terms which represent single node information; pairwise interactions; and in general n-node interactions.The synergy measures introduced in this paper can be easily evaluated by an iterative scaling algorithm, which is a standard procedure in information geometry. |
| first_indexed | 2024-04-13T11:57:33Z |
| format | Article |
| id | doaj.art-e39b2fefaa2d436288843f63073aa9d6 |
| institution | Directory Open Access Journal |
| issn | 2296-9144 |
| language | English |
| last_indexed | 2024-04-13T11:57:33Z |
| publishDate | 2016-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Robotics and AI |
| spelling | doaj.art-e39b2fefaa2d436288843f63073aa9d62022-12-22T02:47:52ZengFrontiers Media S.A.Frontiers in Robotics and AI2296-91442016-01-01210.3389/frobt.2015.00035172511Hierarchical Quantification of Synergy in ChannelsPaolo ePerrone0Nihat eAy1Max Planck Institute for Mathematics in the SciencesMax Planck Institute for Mathematics in the SciencesThe decomposition of channel information into synergies of different order is an open, active problem in the theory of complex systems. Most approaches to the problem are based on information theory, and propose decompositions of mutual information between inputs and outputs in se-veral ways, none of which is generally accepted yet.We propose a new point of view on the topic. We model a multi-input channel as a Markov kernel. We can project the channel onto a series of exponential families which form a hierarchical structure. This is carried out with tools from information geometry, in a way analogous to the projections of probability distributions introduced by Amari. A Pythagorean relation leads naturally to a decomposition of the mutual information between inputs and outputs into terms which represent single node information; pairwise interactions; and in general n-node interactions.The synergy measures introduced in this paper can be easily evaluated by an iterative scaling algorithm, which is a standard procedure in information geometry.http://journal.frontiersin.org/Journal/10.3389/frobt.2015.00035/fullhierarchyinteractionsinformation geometryredundancySynergyprojections |
| spellingShingle | Paolo ePerrone Nihat eAy Hierarchical Quantification of Synergy in Channels Frontiers in Robotics and AI hierarchy interactions information geometry redundancy Synergy projections |
| title | Hierarchical Quantification of Synergy in Channels |
| title_full | Hierarchical Quantification of Synergy in Channels |
| title_fullStr | Hierarchical Quantification of Synergy in Channels |
| title_full_unstemmed | Hierarchical Quantification of Synergy in Channels |
| title_short | Hierarchical Quantification of Synergy in Channels |
| title_sort | hierarchical quantification of synergy in channels |
| topic | hierarchy interactions information geometry redundancy Synergy projections |
| url | http://journal.frontiersin.org/Journal/10.3389/frobt.2015.00035/full |
| work_keys_str_mv | AT paoloeperrone hierarchicalquantificationofsynergyinchannels AT nihateay hierarchicalquantificationofsynergyinchannels |