Stochastic IMT (Insulator-Metal-Transition) Neurons: An Interplay of Thermal and Threshold Noise at Bifurcation
Artificial neural networks can harness stochasticity in multiple ways to enable a vast class of computationally powerful models. Boltzmann machines and other stochastic neural networks have been shown to outperform their deterministic counterparts by allowing dynamical systems to escape local energy...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2018-04-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/article/10.3389/fnins.2018.00210/full |
| _version_ | 1818049881509462016 |
|---|---|
| author | Abhinav Parihar Matthew Jerry Suman Datta Arijit Raychowdhury |
| author_facet | Abhinav Parihar Matthew Jerry Suman Datta Arijit Raychowdhury |
| author_sort | Abhinav Parihar |
| collection | DOAJ |
| description | Artificial neural networks can harness stochasticity in multiple ways to enable a vast class of computationally powerful models. Boltzmann machines and other stochastic neural networks have been shown to outperform their deterministic counterparts by allowing dynamical systems to escape local energy minima. Electronic implementation of such stochastic networks is currently limited to addition of algorithmic noise to digital machines which is inherently inefficient; albeit recent efforts to harness physical noise in devices for stochasticity have shown promise. To succeed in fabricating electronic neuromorphic networks we need experimental evidence of devices with measurable and controllable stochasticity which is complemented with the development of reliable statistical models of such observed stochasticity. Current research literature has sparse evidence of the former and a complete lack of the latter. This motivates the current article where we demonstrate a stochastic neuron using an insulator-metal-transition (IMT) device, based on electrically induced phase-transition, in series with a tunable resistance. We show that an IMT neuron has dynamics similar to a piecewise linear FitzHugh-Nagumo (FHN) neuron and incorporates all characteristics of a spiking neuron in the device phenomena. We experimentally demonstrate spontaneous stochastic spiking along with electrically controllable firing probabilities using Vanadium Dioxide (VO2) based IMT neurons which show a sigmoid-like transfer function. The stochastic spiking is explained by two noise sources - thermal noise and threshold fluctuations, which act as precursors of bifurcation. As such, the IMT neuron is modeled as an Ornstein-Uhlenbeck (OU) process with a fluctuating boundary resulting in transfer curves that closely match experiments. The moments of interspike intervals are calculated analytically by extending the first-passage-time (FPT) models for Ornstein-Uhlenbeck (OU) process to include a fluctuating boundary. We find that the coefficient of variation of interspike intervals depend on the relative proportion of thermal and threshold noise, where threshold noise is the dominant source in the current experimental demonstrations. As one of the first comprehensive studies of a stochastic neuron hardware and its statistical properties, this article would enable efficient implementation of a large class of neuro-mimetic networks and algorithms. |
| first_indexed | 2024-12-10T10:44:37Z |
| format | Article |
| id | doaj.art-7a9141980e514ba9a9fcffef8614f2d8 |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-12-10T10:44:37Z |
| publishDate | 2018-04-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-7a9141980e514ba9a9fcffef8614f2d82022-12-22T01:52:12ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2018-04-011210.3389/fnins.2018.00210305614Stochastic IMT (Insulator-Metal-Transition) Neurons: An Interplay of Thermal and Threshold Noise at BifurcationAbhinav Parihar0Matthew Jerry1Suman Datta2Arijit Raychowdhury3School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, United StatesDepartment of Electrical Engineering, University of Notre Dame, Notre Dame, IN, United StatesDepartment of Electrical Engineering, University of Notre Dame, Notre Dame, IN, United StatesSchool of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, United StatesArtificial neural networks can harness stochasticity in multiple ways to enable a vast class of computationally powerful models. Boltzmann machines and other stochastic neural networks have been shown to outperform their deterministic counterparts by allowing dynamical systems to escape local energy minima. Electronic implementation of such stochastic networks is currently limited to addition of algorithmic noise to digital machines which is inherently inefficient; albeit recent efforts to harness physical noise in devices for stochasticity have shown promise. To succeed in fabricating electronic neuromorphic networks we need experimental evidence of devices with measurable and controllable stochasticity which is complemented with the development of reliable statistical models of such observed stochasticity. Current research literature has sparse evidence of the former and a complete lack of the latter. This motivates the current article where we demonstrate a stochastic neuron using an insulator-metal-transition (IMT) device, based on electrically induced phase-transition, in series with a tunable resistance. We show that an IMT neuron has dynamics similar to a piecewise linear FitzHugh-Nagumo (FHN) neuron and incorporates all characteristics of a spiking neuron in the device phenomena. We experimentally demonstrate spontaneous stochastic spiking along with electrically controllable firing probabilities using Vanadium Dioxide (VO2) based IMT neurons which show a sigmoid-like transfer function. The stochastic spiking is explained by two noise sources - thermal noise and threshold fluctuations, which act as precursors of bifurcation. As such, the IMT neuron is modeled as an Ornstein-Uhlenbeck (OU) process with a fluctuating boundary resulting in transfer curves that closely match experiments. The moments of interspike intervals are calculated analytically by extending the first-passage-time (FPT) models for Ornstein-Uhlenbeck (OU) process to include a fluctuating boundary. We find that the coefficient of variation of interspike intervals depend on the relative proportion of thermal and threshold noise, where threshold noise is the dominant source in the current experimental demonstrations. As one of the first comprehensive studies of a stochastic neuron hardware and its statistical properties, this article would enable efficient implementation of a large class of neuro-mimetic networks and algorithms.http://journal.frontiersin.org/article/10.3389/fnins.2018.00210/fullstochastic neuroninsulator-metal transitionFitzHugh-Nagumo (FHN) neuron modelOrnstein-Uhlenbeck processthreshold noisevanadium-dioxide |
| spellingShingle | Abhinav Parihar Matthew Jerry Suman Datta Arijit Raychowdhury Stochastic IMT (Insulator-Metal-Transition) Neurons: An Interplay of Thermal and Threshold Noise at Bifurcation Frontiers in Neuroscience stochastic neuron insulator-metal transition FitzHugh-Nagumo (FHN) neuron model Ornstein-Uhlenbeck process threshold noise vanadium-dioxide |
| title | Stochastic IMT (Insulator-Metal-Transition) Neurons: An Interplay of Thermal and Threshold Noise at Bifurcation |
| title_full | Stochastic IMT (Insulator-Metal-Transition) Neurons: An Interplay of Thermal and Threshold Noise at Bifurcation |
| title_fullStr | Stochastic IMT (Insulator-Metal-Transition) Neurons: An Interplay of Thermal and Threshold Noise at Bifurcation |
| title_full_unstemmed | Stochastic IMT (Insulator-Metal-Transition) Neurons: An Interplay of Thermal and Threshold Noise at Bifurcation |
| title_short | Stochastic IMT (Insulator-Metal-Transition) Neurons: An Interplay of Thermal and Threshold Noise at Bifurcation |
| title_sort | stochastic imt insulator metal transition neurons an interplay of thermal and threshold noise at bifurcation |
| topic | stochastic neuron insulator-metal transition FitzHugh-Nagumo (FHN) neuron model Ornstein-Uhlenbeck process threshold noise vanadium-dioxide |
| url | http://journal.frontiersin.org/article/10.3389/fnins.2018.00210/full |
| work_keys_str_mv | AT abhinavparihar stochasticimtinsulatormetaltransitionneuronsaninterplayofthermalandthresholdnoiseatbifurcation AT matthewjerry stochasticimtinsulatormetaltransitionneuronsaninterplayofthermalandthresholdnoiseatbifurcation AT sumandatta stochasticimtinsulatormetaltransitionneuronsaninterplayofthermalandthresholdnoiseatbifurcation AT arijitraychowdhury stochasticimtinsulatormetaltransitionneuronsaninterplayofthermalandthresholdnoiseatbifurcation |