SuperSpike: supervised learning in multilayer spiking neural networks
A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in vivo, as well as how we can instantiate such capabilities in artificial spiking...
| Main Authors: | , |
|---|---|
| Format: | Journal article |
| Published: |
Massachusetts Institute of Technology Press
2018
|
| _version_ | 1826263109365399552 |
|---|---|
| author | Zenke, F Ganguli, S |
| author_facet | Zenke, F Ganguli, S |
| author_sort | Zenke, F |
| collection | OXFORD |
| description | A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in silico. Here we revisit the problem of supervised learning in temporally coding multilayer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three-factor learning rule capable of training multilayer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric, and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike time patterns. |
| first_indexed | 2024-03-06T19:46:29Z |
| format | Journal article |
| id | oxford-uuid:22779ec3-6911-4135-8d09-67a56161fb7e |
| institution | University of Oxford |
| last_indexed | 2024-03-06T19:46:29Z |
| publishDate | 2018 |
| publisher | Massachusetts Institute of Technology Press |
| record_format | dspace |
| spelling | oxford-uuid:22779ec3-6911-4135-8d09-67a56161fb7e2022-03-26T11:39:01ZSuperSpike: supervised learning in multilayer spiking neural networksJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:22779ec3-6911-4135-8d09-67a56161fb7eSymplectic Elements at OxfordMassachusetts Institute of Technology Press2018Zenke, FGanguli, SA vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in silico. Here we revisit the problem of supervised learning in temporally coding multilayer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three-factor learning rule capable of training multilayer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric, and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike time patterns. |
| spellingShingle | Zenke, F Ganguli, S SuperSpike: supervised learning in multilayer spiking neural networks |
| title | SuperSpike: supervised learning in multilayer spiking neural networks |
| title_full | SuperSpike: supervised learning in multilayer spiking neural networks |
| title_fullStr | SuperSpike: supervised learning in multilayer spiking neural networks |
| title_full_unstemmed | SuperSpike: supervised learning in multilayer spiking neural networks |
| title_short | SuperSpike: supervised learning in multilayer spiking neural networks |
| title_sort | superspike supervised learning in multilayer spiking neural networks |
| work_keys_str_mv | AT zenkef superspikesupervisedlearninginmultilayerspikingneuralnetworks AT gangulis superspikesupervisedlearninginmultilayerspikingneuralnetworks |