Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms
Recurrent neural networks are currently subject to intensive research efforts to solve temporal computing problems. Neuromorphic processors (NPs), composed of networked neuron and synapse circuit models, natively compute in time and offer an ultralow power solution particularly suited to emerging te...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
AIP Publishing LLC
2019-08-01
|
| Series: | APL Materials |
| Online Access: | http://dx.doi.org/10.1063/1.5108663 |
| _version_ | 1818198690681061376 |
|---|---|
| author | Thomas Dalgaty Melika Payvand Filippo Moro Denys R. B. Ly Florian Pebay-Peyroula Jerome Casas Giacomo Indiveri Elisa Vianello |
| author_facet | Thomas Dalgaty Melika Payvand Filippo Moro Denys R. B. Ly Florian Pebay-Peyroula Jerome Casas Giacomo Indiveri Elisa Vianello |
| author_sort | Thomas Dalgaty |
| collection | DOAJ |
| description | Recurrent neural networks are currently subject to intensive research efforts to solve temporal computing problems. Neuromorphic processors (NPs), composed of networked neuron and synapse circuit models, natively compute in time and offer an ultralow power solution particularly suited to emerging temporal edge-computing applications (wearable medical devices, for example). The most significant roadblock to addressing useful problems with neuromorphic hardware is the difficulty in maintaining healthy network dynamics in recurrent neural networks. In animal nervous systems, this is achieved via a multitude of adaptive homeostatic mechanisms which act over multiple time scales to counteract network instability induced via drift, component failure, or learning processes such as spike-timing dependent plasticity. One such mechanism is neuronal intrinsic plasticity (IP) where a neuron adapts its parameters which govern its excitability to fire around a target rate. The approach employed in state of the art NPs, based on a central volatile memory remotely setting model parameters, critically constrains parameter variety and bandwidth rendering realization of these essential mechanisms impossible. This paper demonstrates how reconfigurable nonvolatile resistive memories can be incorporated into neuron and synapse circuits allowing memory to be truly colocalized with the computational units in the computing fabric and facilitating the realization of massively parallel local plasticity mechanisms in neuromorphic hardware. Exploiting nonconventional programming operations of HfO2 based RRAM (stochastic SET and the RESET random variable), we propose a technologically plausible IP algorithm and demonstrate its use in the case of a recurrent neural network topology whereby the system self-organizes to sustain stable and healthy network dynamics around a target firing rate. |
| first_indexed | 2024-12-12T02:09:53Z |
| format | Article |
| id | doaj.art-b467842f2b9e42fd976586eeca9d6a38 |
| institution | Directory Open Access Journal |
| issn | 2166-532X |
| language | English |
| last_indexed | 2024-12-12T02:09:53Z |
| publishDate | 2019-08-01 |
| publisher | AIP Publishing LLC |
| record_format | Article |
| series | APL Materials |
| spelling | doaj.art-b467842f2b9e42fd976586eeca9d6a382022-12-22T00:41:55ZengAIP Publishing LLCAPL Materials2166-532X2019-08-0178081125081125-1210.1063/1.5108663008908APMHybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanismsThomas Dalgaty0Melika Payvand1Filippo Moro2Denys R. B. Ly3Florian Pebay-Peyroula4Jerome Casas5Giacomo Indiveri6Elisa Vianello7CEA-Leti, Grenoble, FranceUniversity of Zurich and ETH Zurich, Zurich, SwitzerlandPolitecnico di Torino, Torino, ItalyCEA-Leti, Grenoble, FranceCEA-Leti, Grenoble, FranceUniversity of Tours, Tours, FranceUniversity of Zurich and ETH Zurich, Zurich, SwitzerlandCEA-Leti, Grenoble, FranceRecurrent neural networks are currently subject to intensive research efforts to solve temporal computing problems. Neuromorphic processors (NPs), composed of networked neuron and synapse circuit models, natively compute in time and offer an ultralow power solution particularly suited to emerging temporal edge-computing applications (wearable medical devices, for example). The most significant roadblock to addressing useful problems with neuromorphic hardware is the difficulty in maintaining healthy network dynamics in recurrent neural networks. In animal nervous systems, this is achieved via a multitude of adaptive homeostatic mechanisms which act over multiple time scales to counteract network instability induced via drift, component failure, or learning processes such as spike-timing dependent plasticity. One such mechanism is neuronal intrinsic plasticity (IP) where a neuron adapts its parameters which govern its excitability to fire around a target rate. The approach employed in state of the art NPs, based on a central volatile memory remotely setting model parameters, critically constrains parameter variety and bandwidth rendering realization of these essential mechanisms impossible. This paper demonstrates how reconfigurable nonvolatile resistive memories can be incorporated into neuron and synapse circuits allowing memory to be truly colocalized with the computational units in the computing fabric and facilitating the realization of massively parallel local plasticity mechanisms in neuromorphic hardware. Exploiting nonconventional programming operations of HfO2 based RRAM (stochastic SET and the RESET random variable), we propose a technologically plausible IP algorithm and demonstrate its use in the case of a recurrent neural network topology whereby the system self-organizes to sustain stable and healthy network dynamics around a target firing rate.http://dx.doi.org/10.1063/1.5108663 |
| spellingShingle | Thomas Dalgaty Melika Payvand Filippo Moro Denys R. B. Ly Florian Pebay-Peyroula Jerome Casas Giacomo Indiveri Elisa Vianello Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms APL Materials |
| title | Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms |
| title_full | Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms |
| title_fullStr | Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms |
| title_full_unstemmed | Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms |
| title_short | Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms |
| title_sort | hybrid neuromorphic circuits exploiting non conventional properties of rram for massively parallel local plasticity mechanisms |
| url | http://dx.doi.org/10.1063/1.5108663 |
| work_keys_str_mv | AT thomasdalgaty hybridneuromorphiccircuitsexploitingnonconventionalpropertiesofrramformassivelyparallellocalplasticitymechanisms AT melikapayvand hybridneuromorphiccircuitsexploitingnonconventionalpropertiesofrramformassivelyparallellocalplasticitymechanisms AT filippomoro hybridneuromorphiccircuitsexploitingnonconventionalpropertiesofrramformassivelyparallellocalplasticitymechanisms AT denysrbly hybridneuromorphiccircuitsexploitingnonconventionalpropertiesofrramformassivelyparallellocalplasticitymechanisms AT florianpebaypeyroula hybridneuromorphiccircuitsexploitingnonconventionalpropertiesofrramformassivelyparallellocalplasticitymechanisms AT jeromecasas hybridneuromorphiccircuitsexploitingnonconventionalpropertiesofrramformassivelyparallellocalplasticitymechanisms AT giacomoindiveri hybridneuromorphiccircuitsexploitingnonconventionalpropertiesofrramformassivelyparallellocalplasticitymechanisms AT elisavianello hybridneuromorphiccircuitsexploitingnonconventionalpropertiesofrramformassivelyparallellocalplasticitymechanisms |