Optimal Input Representation in Neural Systems at the Edge of Chaos
Shedding light on how biological systems represent, process and store information in noisy environments is a key and challenging goal. A stimulating, though controversial, hypothesis poses that operating in dynamical regimes near the edge of a phase transition, i.e., at criticality or the “edge of c...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Biology |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-7737/10/8/702 |
| _version_ | 1827685890723414016 |
|---|---|
| author | Guillermo B. Morales Miguel A. Muñoz |
| author_facet | Guillermo B. Morales Miguel A. Muñoz |
| author_sort | Guillermo B. Morales |
| collection | DOAJ |
| description | Shedding light on how biological systems represent, process and store information in noisy environments is a key and challenging goal. A stimulating, though controversial, hypothesis poses that operating in dynamical regimes near the edge of a phase transition, i.e., at criticality or the “edge of chaos”, can provide information-processing living systems with important operational advantages, creating, e.g., an optimal trade-off between robustness and flexibility. Here, we elaborate on a recent theoretical result, which establishes that the spectrum of covariance matrices of neural networks representing complex inputs in a robust way needs to decay as a power-law of the rank, with an exponent close to unity, a result that has been indeed experimentally verified in neurons of the mouse visual cortex. Aimed at understanding and mimicking these results, we construct an artificial neural network and train it to classify images. We find that the best performance in such a task is obtained when the network operates near the critical point, at which the eigenspectrum of the covariance matrix follows the very same statistics as actual neurons do. Thus, we conclude that operating near criticality can also have—besides the usually alleged virtues—the advantage of allowing for flexible, robust and efficient input representations. |
| first_indexed | 2024-03-10T09:00:49Z |
| format | Article |
| id | doaj.art-d878e344881145f189f1cb18e161308f |
| institution | Directory Open Access Journal |
| issn | 2079-7737 |
| language | English |
| last_indexed | 2024-03-10T09:00:49Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Biology |
| spelling | doaj.art-d878e344881145f189f1cb18e161308f2023-11-22T06:49:19ZengMDPI AGBiology2079-77372021-07-0110870210.3390/biology10080702Optimal Input Representation in Neural Systems at the Edge of ChaosGuillermo B. Morales0Miguel A. Muñoz1Departamento de Electromagnetismo y Física de la Materia, Instituto Carlos I de Física Teórica y Computacional, Universidad de Granada, E-18071 Granada, SpainDepartamento de Electromagnetismo y Física de la Materia, Instituto Carlos I de Física Teórica y Computacional, Universidad de Granada, E-18071 Granada, SpainShedding light on how biological systems represent, process and store information in noisy environments is a key and challenging goal. A stimulating, though controversial, hypothesis poses that operating in dynamical regimes near the edge of a phase transition, i.e., at criticality or the “edge of chaos”, can provide information-processing living systems with important operational advantages, creating, e.g., an optimal trade-off between robustness and flexibility. Here, we elaborate on a recent theoretical result, which establishes that the spectrum of covariance matrices of neural networks representing complex inputs in a robust way needs to decay as a power-law of the rank, with an exponent close to unity, a result that has been indeed experimentally verified in neurons of the mouse visual cortex. Aimed at understanding and mimicking these results, we construct an artificial neural network and train it to classify images. We find that the best performance in such a task is obtained when the network operates near the critical point, at which the eigenspectrum of the covariance matrix follows the very same statistics as actual neurons do. Thus, we conclude that operating near criticality can also have—besides the usually alleged virtues—the advantage of allowing for flexible, robust and efficient input representations.https://www.mdpi.com/2079-7737/10/8/702information processinginput representationneural networkscriticality hypothesisedge of chaosreservoir computing |
| spellingShingle | Guillermo B. Morales Miguel A. Muñoz Optimal Input Representation in Neural Systems at the Edge of Chaos Biology information processing input representation neural networks criticality hypothesis edge of chaos reservoir computing |
| title | Optimal Input Representation in Neural Systems at the Edge of Chaos |
| title_full | Optimal Input Representation in Neural Systems at the Edge of Chaos |
| title_fullStr | Optimal Input Representation in Neural Systems at the Edge of Chaos |
| title_full_unstemmed | Optimal Input Representation in Neural Systems at the Edge of Chaos |
| title_short | Optimal Input Representation in Neural Systems at the Edge of Chaos |
| title_sort | optimal input representation in neural systems at the edge of chaos |
| topic | information processing input representation neural networks criticality hypothesis edge of chaos reservoir computing |
| url | https://www.mdpi.com/2079-7737/10/8/702 |
| work_keys_str_mv | AT guillermobmorales optimalinputrepresentationinneuralsystemsattheedgeofchaos AT miguelamunoz optimalinputrepresentationinneuralsystemsattheedgeofchaos |