Digital Biologically Plausible Implementation of Binarized Neural Networks With Differential Hafnium Oxide Resistive Memory Arrays
The brain performs intelligent tasks with extremely low energy consumption. This work takes its inspiration from two strategies used by the brain to achieve this energy efficiency: the absence of separation between computing and memory functions and reliance on low-precision computation. The emergen...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2020-01-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fnins.2019.01383/full |
| _version_ | 1818849131748130816 |
|---|---|
| author | Tifenn Hirtzlin Marc Bocquet Bogdan Penkovsky Jacques-Olivier Klein Etienne Nowak Elisa Vianello Jean-Michel Portal Damien Querlioz |
| author_facet | Tifenn Hirtzlin Marc Bocquet Bogdan Penkovsky Jacques-Olivier Klein Etienne Nowak Elisa Vianello Jean-Michel Portal Damien Querlioz |
| author_sort | Tifenn Hirtzlin |
| collection | DOAJ |
| description | The brain performs intelligent tasks with extremely low energy consumption. This work takes its inspiration from two strategies used by the brain to achieve this energy efficiency: the absence of separation between computing and memory functions and reliance on low-precision computation. The emergence of resistive memory technologies indeed provides an opportunity to tightly co-integrate logic and memory in hardware. In parallel, the recently proposed concept of a Binarized Neural Network, where multiplications are replaced by exclusive NOR (XNOR) logic gates, offers a way to implement artificial intelligence using very low precision computation. In this work, we therefore propose a strategy for implementing low-energy Binarized Neural Networks that employs brain-inspired concepts while retaining the energy benefits of digital electronics. We design, fabricate, and test a memory array, including periphery and sensing circuits, that is optimized for this in-memory computing scheme. Our circuit employs hafnium oxide resistive memory integrated in the back end of line of a 130-nm CMOS process, in a two-transistor, two-resistor cell, which allows the exclusive NOR operations of the neural network to be performed directly within the sense amplifiers. We show, based on extensive electrical measurements, that our design allows a reduction in the number of bit errors on the synaptic weights without the use of formal error-correcting codes. We design a whole system using this memory array. We show on standard machine learning tasks (MNIST, CIFAR-10, ImageNet, and an ECG task) that the system has inherent resilience to bit errors. We evidence that its energy consumption is attractive compared to more standard approaches and that it can use memory devices in regimes where they exhibit particularly low programming energy and high endurance. We conclude the work by discussing how it associates biologically plausible ideas with more traditional digital electronics concepts. |
| first_indexed | 2024-12-19T06:28:22Z |
| format | Article |
| id | doaj.art-dbd1743fa64545fea508526d5b5e607c |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-12-19T06:28:22Z |
| publishDate | 2020-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-dbd1743fa64545fea508526d5b5e607c2022-12-21T20:32:28ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2020-01-011310.3389/fnins.2019.01383489982Digital Biologically Plausible Implementation of Binarized Neural Networks With Differential Hafnium Oxide Resistive Memory ArraysTifenn Hirtzlin0Marc Bocquet1Bogdan Penkovsky2Jacques-Olivier Klein3Etienne Nowak4Elisa Vianello5Jean-Michel Portal6Damien Querlioz7C2N, Univ Paris-Sud, Université Paris-Saclay, CNRS, Palaiseau, FranceAix Marseille Univ, Université de Toulon, CNRS, IM2NP, Marseille, FranceC2N, Univ Paris-Sud, Université Paris-Saclay, CNRS, Palaiseau, FranceC2N, Univ Paris-Sud, Université Paris-Saclay, CNRS, Palaiseau, FranceCEA, LETI, Grenoble, FranceCEA, LETI, Grenoble, FranceAix Marseille Univ, Université de Toulon, CNRS, IM2NP, Marseille, FranceC2N, Univ Paris-Sud, Université Paris-Saclay, CNRS, Palaiseau, FranceThe brain performs intelligent tasks with extremely low energy consumption. This work takes its inspiration from two strategies used by the brain to achieve this energy efficiency: the absence of separation between computing and memory functions and reliance on low-precision computation. The emergence of resistive memory technologies indeed provides an opportunity to tightly co-integrate logic and memory in hardware. In parallel, the recently proposed concept of a Binarized Neural Network, where multiplications are replaced by exclusive NOR (XNOR) logic gates, offers a way to implement artificial intelligence using very low precision computation. In this work, we therefore propose a strategy for implementing low-energy Binarized Neural Networks that employs brain-inspired concepts while retaining the energy benefits of digital electronics. We design, fabricate, and test a memory array, including periphery and sensing circuits, that is optimized for this in-memory computing scheme. Our circuit employs hafnium oxide resistive memory integrated in the back end of line of a 130-nm CMOS process, in a two-transistor, two-resistor cell, which allows the exclusive NOR operations of the neural network to be performed directly within the sense amplifiers. We show, based on extensive electrical measurements, that our design allows a reduction in the number of bit errors on the synaptic weights without the use of formal error-correcting codes. We design a whole system using this memory array. We show on standard machine learning tasks (MNIST, CIFAR-10, ImageNet, and an ECG task) that the system has inherent resilience to bit errors. We evidence that its energy consumption is attractive compared to more standard approaches and that it can use memory devices in regimes where they exhibit particularly low programming energy and high endurance. We conclude the work by discussing how it associates biologically plausible ideas with more traditional digital electronics concepts.https://www.frontiersin.org/article/10.3389/fnins.2019.01383/fullbinarized neural networksresistive memorymemristorin-memory computingbiologically plausible digital electronicsASICs |
| spellingShingle | Tifenn Hirtzlin Marc Bocquet Bogdan Penkovsky Jacques-Olivier Klein Etienne Nowak Elisa Vianello Jean-Michel Portal Damien Querlioz Digital Biologically Plausible Implementation of Binarized Neural Networks With Differential Hafnium Oxide Resistive Memory Arrays Frontiers in Neuroscience binarized neural networks resistive memory memristor in-memory computing biologically plausible digital electronics ASICs |
| title | Digital Biologically Plausible Implementation of Binarized Neural Networks With Differential Hafnium Oxide Resistive Memory Arrays |
| title_full | Digital Biologically Plausible Implementation of Binarized Neural Networks With Differential Hafnium Oxide Resistive Memory Arrays |
| title_fullStr | Digital Biologically Plausible Implementation of Binarized Neural Networks With Differential Hafnium Oxide Resistive Memory Arrays |
| title_full_unstemmed | Digital Biologically Plausible Implementation of Binarized Neural Networks With Differential Hafnium Oxide Resistive Memory Arrays |
| title_short | Digital Biologically Plausible Implementation of Binarized Neural Networks With Differential Hafnium Oxide Resistive Memory Arrays |
| title_sort | digital biologically plausible implementation of binarized neural networks with differential hafnium oxide resistive memory arrays |
| topic | binarized neural networks resistive memory memristor in-memory computing biologically plausible digital electronics ASICs |
| url | https://www.frontiersin.org/article/10.3389/fnins.2019.01383/full |
| work_keys_str_mv | AT tifennhirtzlin digitalbiologicallyplausibleimplementationofbinarizedneuralnetworkswithdifferentialhafniumoxideresistivememoryarrays AT marcbocquet digitalbiologicallyplausibleimplementationofbinarizedneuralnetworkswithdifferentialhafniumoxideresistivememoryarrays AT bogdanpenkovsky digitalbiologicallyplausibleimplementationofbinarizedneuralnetworkswithdifferentialhafniumoxideresistivememoryarrays AT jacquesolivierklein digitalbiologicallyplausibleimplementationofbinarizedneuralnetworkswithdifferentialhafniumoxideresistivememoryarrays AT etiennenowak digitalbiologicallyplausibleimplementationofbinarizedneuralnetworkswithdifferentialhafniumoxideresistivememoryarrays AT elisavianello digitalbiologicallyplausibleimplementationofbinarizedneuralnetworkswithdifferentialhafniumoxideresistivememoryarrays AT jeanmichelportal digitalbiologicallyplausibleimplementationofbinarizedneuralnetworkswithdifferentialhafniumoxideresistivememoryarrays AT damienquerlioz digitalbiologicallyplausibleimplementationofbinarizedneuralnetworkswithdifferentialhafniumoxideresistivememoryarrays |