Information Anatomy of Stochastic Equilibria
A stochastic nonlinear dynamical system generates information, as measured by its entropy rate. Some—the ephemeral information—is dissipated and some—the bound information—is actively stored and so affects future behavior. We derive analytic expressions for the ephemeral and bound information in the...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2014-08-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1099-4300/16/9/4713 |
| _version_ | 1811187114618912768 |
|---|---|
| author | Sarah Marzen James P. Crutchfield |
| author_facet | Sarah Marzen James P. Crutchfield |
| author_sort | Sarah Marzen |
| collection | DOAJ |
| description | A stochastic nonlinear dynamical system generates information, as measured by its entropy rate. Some—the ephemeral information—is dissipated and some—the bound information—is actively stored and so affects future behavior. We derive analytic expressions for the ephemeral and bound information in the limit of infinitesimal time discretization for two classical systems that exhibit dynamical equilibria: first-order Langevin equations (i) where the drift is the gradient of an analytic potential function and the diffusion matrix is invertible and (ii) with a linear drift term (Ornstein–Uhlenbeck), but a noninvertible diffusion matrix. In both cases, the bound information is sensitive to the drift and diffusion, while the ephemeral information is sensitive only to the diffusion matrix and not to the drift. Notably, this information anatomy changes discontinuously as any of the diffusion coefficients vanishes, indicating that it is very sensitive to the noise structure. We then calculate the information anatomy of the stochastic cusp catastrophe and of particles diffusing in a heat bath in the overdamped limit, both examples of stochastic gradient descent on a potential landscape. Finally, we use our methods to calculate and compare approximations for the time-local predictive information for adaptive agents. |
| first_indexed | 2024-04-11T13:56:44Z |
| format | Article |
| id | doaj.art-131d22e9d8ea402da9ec938d1d1e6e18 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-04-11T13:56:44Z |
| publishDate | 2014-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-131d22e9d8ea402da9ec938d1d1e6e182022-12-22T04:20:13ZengMDPI AGEntropy1099-43002014-08-011694713474810.3390/e16094713e16094713Information Anatomy of Stochastic EquilibriaSarah Marzen0James P. Crutchfield1Department of Physics, University of California at Berkeley, Berkeley, CA 94720, USAComplexity Sciences Center, Department of Physics, University of California at Davis, One Shields Avenue, Davis, CA 95616, USAA stochastic nonlinear dynamical system generates information, as measured by its entropy rate. Some—the ephemeral information—is dissipated and some—the bound information—is actively stored and so affects future behavior. We derive analytic expressions for the ephemeral and bound information in the limit of infinitesimal time discretization for two classical systems that exhibit dynamical equilibria: first-order Langevin equations (i) where the drift is the gradient of an analytic potential function and the diffusion matrix is invertible and (ii) with a linear drift term (Ornstein–Uhlenbeck), but a noninvertible diffusion matrix. In both cases, the bound information is sensitive to the drift and diffusion, while the ephemeral information is sensitive only to the diffusion matrix and not to the drift. Notably, this information anatomy changes discontinuously as any of the diffusion coefficients vanishes, indicating that it is very sensitive to the noise structure. We then calculate the information anatomy of the stochastic cusp catastrophe and of particles diffusing in a heat bath in the overdamped limit, both examples of stochastic gradient descent on a potential landscape. Finally, we use our methods to calculate and compare approximations for the time-local predictive information for adaptive agents.http://www.mdpi.com/1099-4300/16/9/4713Langevin equationentropy rateephemeral informationbound informationtime-local predictive information |
| spellingShingle | Sarah Marzen James P. Crutchfield Information Anatomy of Stochastic Equilibria Entropy Langevin equation entropy rate ephemeral information bound information time-local predictive information |
| title | Information Anatomy of Stochastic Equilibria |
| title_full | Information Anatomy of Stochastic Equilibria |
| title_fullStr | Information Anatomy of Stochastic Equilibria |
| title_full_unstemmed | Information Anatomy of Stochastic Equilibria |
| title_short | Information Anatomy of Stochastic Equilibria |
| title_sort | information anatomy of stochastic equilibria |
| topic | Langevin equation entropy rate ephemeral information bound information time-local predictive information |
| url | http://www.mdpi.com/1099-4300/16/9/4713 |
| work_keys_str_mv | AT sarahmarzen informationanatomyofstochasticequilibria AT jamespcrutchfield informationanatomyofstochasticequilibria |