Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks
Abstract Safety-critical sensory applications, like medical diagnosis, demand accurate decisions from limited, noisy data. Bayesian neural networks excel at such tasks, offering predictive uncertainty assessment. However, because of their probabilistic nature, they are computationally intensive. An...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2023-11-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-023-43317-9 |
| _version_ | 1797452107359453184 |
|---|---|
| author | Djohan Bonnet Tifenn Hirtzlin Atreya Majumdar Thomas Dalgaty Eduardo Esmanhotto Valentina Meli Niccolo Castellani Simon Martin Jean-François Nodin Guillaume Bourgeois Jean-Michel Portal Damien Querlioz Elisa Vianello |
| author_facet | Djohan Bonnet Tifenn Hirtzlin Atreya Majumdar Thomas Dalgaty Eduardo Esmanhotto Valentina Meli Niccolo Castellani Simon Martin Jean-François Nodin Guillaume Bourgeois Jean-Michel Portal Damien Querlioz Elisa Vianello |
| author_sort | Djohan Bonnet |
| collection | DOAJ |
| description | Abstract Safety-critical sensory applications, like medical diagnosis, demand accurate decisions from limited, noisy data. Bayesian neural networks excel at such tasks, offering predictive uncertainty assessment. However, because of their probabilistic nature, they are computationally intensive. An innovative solution utilizes memristors’ inherent probabilistic nature to implement Bayesian neural networks. However, when using memristors, statistical effects follow the laws of device physics, whereas in Bayesian neural networks, those effects can take arbitrary shapes. This work overcome this difficulty by adopting a variational inference training augmented by a “technological loss”, incorporating memristor physics. This technique enabled programming a Bayesian neural network on 75 crossbar arrays of 1,024 memristors, incorporating CMOS periphery for in-memory computing. The experimental neural network classified heartbeats with high accuracy, and estimated the certainty of its predictions. The results reveal orders-of-magnitude improvement in inference energy efficiency compared to a microcontroller or an embedded graphics processing unit performing the same task. |
| first_indexed | 2024-03-09T15:04:00Z |
| format | Article |
| id | doaj.art-71a0f04b513342a69e2eef0f95dfe8f1 |
| institution | Directory Open Access Journal |
| issn | 2041-1723 |
| language | English |
| last_indexed | 2024-03-09T15:04:00Z |
| publishDate | 2023-11-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj.art-71a0f04b513342a69e2eef0f95dfe8f12023-11-26T13:45:23ZengNature PortfolioNature Communications2041-17232023-11-0114111310.1038/s41467-023-43317-9Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networksDjohan Bonnet0Tifenn Hirtzlin1Atreya Majumdar2Thomas Dalgaty3Eduardo Esmanhotto4Valentina Meli5Niccolo Castellani6Simon Martin7Jean-François Nodin8Guillaume Bourgeois9Jean-Michel Portal10Damien Querlioz11Elisa Vianello12Université Grenoble Alpes, CEA, LETIUniversité Grenoble Alpes, CEA, LETIUniversité Paris-Saclay, CNRS, Centre de Nanosciences et de NanotechnologiesUniversité Grenoble Alpes, CEA, LISTUniversité Grenoble Alpes, CEA, LETIUniversité Grenoble Alpes, CEA, LETIUniversité Grenoble Alpes, CEA, LETIUniversité Grenoble Alpes, CEA, LETIUniversité Grenoble Alpes, CEA, LETIUniversité Grenoble Alpes, CEA, LETIAix-Marseille Université, CNRS, Institut Matériaux Microélectronique Nanosciences de ProvenceUniversité Paris-Saclay, CNRS, Centre de Nanosciences et de NanotechnologiesUniversité Grenoble Alpes, CEA, LETIAbstract Safety-critical sensory applications, like medical diagnosis, demand accurate decisions from limited, noisy data. Bayesian neural networks excel at such tasks, offering predictive uncertainty assessment. However, because of their probabilistic nature, they are computationally intensive. An innovative solution utilizes memristors’ inherent probabilistic nature to implement Bayesian neural networks. However, when using memristors, statistical effects follow the laws of device physics, whereas in Bayesian neural networks, those effects can take arbitrary shapes. This work overcome this difficulty by adopting a variational inference training augmented by a “technological loss”, incorporating memristor physics. This technique enabled programming a Bayesian neural network on 75 crossbar arrays of 1,024 memristors, incorporating CMOS periphery for in-memory computing. The experimental neural network classified heartbeats with high accuracy, and estimated the certainty of its predictions. The results reveal orders-of-magnitude improvement in inference energy efficiency compared to a microcontroller or an embedded graphics processing unit performing the same task.https://doi.org/10.1038/s41467-023-43317-9 |
| spellingShingle | Djohan Bonnet Tifenn Hirtzlin Atreya Majumdar Thomas Dalgaty Eduardo Esmanhotto Valentina Meli Niccolo Castellani Simon Martin Jean-François Nodin Guillaume Bourgeois Jean-Michel Portal Damien Querlioz Elisa Vianello Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks Nature Communications |
| title | Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks |
| title_full | Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks |
| title_fullStr | Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks |
| title_full_unstemmed | Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks |
| title_short | Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks |
| title_sort | bringing uncertainty quantification to the extreme edge with memristor based bayesian neural networks |
| url | https://doi.org/10.1038/s41467-023-43317-9 |
| work_keys_str_mv | AT djohanbonnet bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT tifennhirtzlin bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT atreyamajumdar bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT thomasdalgaty bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT eduardoesmanhotto bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT valentinameli bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT niccolocastellani bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT simonmartin bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT jeanfrancoisnodin bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT guillaumebourgeois bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT jeanmichelportal bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT damienquerlioz bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks AT elisavianello bringinguncertaintyquantificationtotheextremeedgewithmemristorbasedbayesianneuralnetworks |