Direct training high-performance spiking neural networks for object recognition and detection
IntroductionThe spiking neural network (SNN) is a bionic model that is energy-efficient when implemented on neuromorphic hardwares. The non-differentiability of the spiking signals and the complicated neural dynamics make direct training of high-performance SNNs a great challenge. There are numerous...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-08-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnins.2023.1229951/full |
| _version_ | 1797751892255703040 |
|---|---|
| author | Hong Zhang Yang Li Bin He Xiongfei Fan Yue Wang Yu Zhang Yu Zhang |
| author_facet | Hong Zhang Yang Li Bin He Xiongfei Fan Yue Wang Yu Zhang Yu Zhang |
| author_sort | Hong Zhang |
| collection | DOAJ |
| description | IntroductionThe spiking neural network (SNN) is a bionic model that is energy-efficient when implemented on neuromorphic hardwares. The non-differentiability of the spiking signals and the complicated neural dynamics make direct training of high-performance SNNs a great challenge. There are numerous crucial issues to explore for the deployment of direct training SNNs, such as gradient vanishing and explosion, spiking signal decoding, and applications in upstream tasks.MethodsTo address gradient vanishing, we introduce a binary selection gate into the basic residual block and propose spiking gate (SG) ResNet to implement residual learning in SNNs. We propose two appropriate representations of the gate signal and verify that SG ResNet can overcome gradient vanishing or explosion by analyzing the gradient backpropagation. For the spiking signal decoding, a better decoding scheme than rate coding is achieved by our attention spike decoder (ASD), which dynamically assigns weights to spiking signals along the temporal, channel, and spatial dimensions.Results and discussionThe SG ResNet and ASD modules are evaluated on multiple object recognition datasets, including the static ImageNet, CIFAR-100, CIFAR-10, and neuromorphic DVS-CIFAR10 datasets. Superior accuracy is demonstrated with a tiny simulation time step of four, specifically 94.52% top-1 accuracy on CIFAR-10 and 75.64% top-1 accuracy on CIFAR-100. Spiking RetinaNet is proposed using SG ResNet as the backbone and ASD module for information decoding as the first direct-training hybrid SNN-ANN detector for RGB images. Spiking RetinaNet with a SG ResNet34 backbone achieves an mAP of 0.296 on the object detection dataset MSCOCO. |
| first_indexed | 2024-03-12T16:56:17Z |
| format | Article |
| id | doaj.art-fd39f0a78e0b4a0cb788e7bba2fc55b3 |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-03-12T16:56:17Z |
| publishDate | 2023-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-fd39f0a78e0b4a0cb788e7bba2fc55b32023-08-08T07:43:36ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2023-08-011710.3389/fnins.2023.12299511229951Direct training high-performance spiking neural networks for object recognition and detectionHong Zhang0Yang Li1Bin He2Xiongfei Fan3Yue Wang4Yu Zhang5Yu Zhang6State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou, ChinaState Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou, ChinaState Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou, ChinaState Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou, ChinaState Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou, ChinaState Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University, Hangzhou, ChinaKey Laboratory of Collaborative Sensing and Autonomous Unmanned Systems of Zhejiang Province, Hangzhou, ChinaIntroductionThe spiking neural network (SNN) is a bionic model that is energy-efficient when implemented on neuromorphic hardwares. The non-differentiability of the spiking signals and the complicated neural dynamics make direct training of high-performance SNNs a great challenge. There are numerous crucial issues to explore for the deployment of direct training SNNs, such as gradient vanishing and explosion, spiking signal decoding, and applications in upstream tasks.MethodsTo address gradient vanishing, we introduce a binary selection gate into the basic residual block and propose spiking gate (SG) ResNet to implement residual learning in SNNs. We propose two appropriate representations of the gate signal and verify that SG ResNet can overcome gradient vanishing or explosion by analyzing the gradient backpropagation. For the spiking signal decoding, a better decoding scheme than rate coding is achieved by our attention spike decoder (ASD), which dynamically assigns weights to spiking signals along the temporal, channel, and spatial dimensions.Results and discussionThe SG ResNet and ASD modules are evaluated on multiple object recognition datasets, including the static ImageNet, CIFAR-100, CIFAR-10, and neuromorphic DVS-CIFAR10 datasets. Superior accuracy is demonstrated with a tiny simulation time step of four, specifically 94.52% top-1 accuracy on CIFAR-10 and 75.64% top-1 accuracy on CIFAR-100. Spiking RetinaNet is proposed using SG ResNet as the backbone and ASD module for information decoding as the first direct-training hybrid SNN-ANN detector for RGB images. Spiking RetinaNet with a SG ResNet34 backbone achieves an mAP of 0.296 on the object detection dataset MSCOCO.https://www.frontiersin.org/articles/10.3389/fnins.2023.1229951/fullspiking neural networksgate residual learningattention spike decoderspiking RetinaNetobject recognitionobject detection |
| spellingShingle | Hong Zhang Yang Li Bin He Xiongfei Fan Yue Wang Yu Zhang Yu Zhang Direct training high-performance spiking neural networks for object recognition and detection Frontiers in Neuroscience spiking neural networks gate residual learning attention spike decoder spiking RetinaNet object recognition object detection |
| title | Direct training high-performance spiking neural networks for object recognition and detection |
| title_full | Direct training high-performance spiking neural networks for object recognition and detection |
| title_fullStr | Direct training high-performance spiking neural networks for object recognition and detection |
| title_full_unstemmed | Direct training high-performance spiking neural networks for object recognition and detection |
| title_short | Direct training high-performance spiking neural networks for object recognition and detection |
| title_sort | direct training high performance spiking neural networks for object recognition and detection |
| topic | spiking neural networks gate residual learning attention spike decoder spiking RetinaNet object recognition object detection |
| url | https://www.frontiersin.org/articles/10.3389/fnins.2023.1229951/full |
| work_keys_str_mv | AT hongzhang directtraininghighperformancespikingneuralnetworksforobjectrecognitionanddetection AT yangli directtraininghighperformancespikingneuralnetworksforobjectrecognitionanddetection AT binhe directtraininghighperformancespikingneuralnetworksforobjectrecognitionanddetection AT xiongfeifan directtraininghighperformancespikingneuralnetworksforobjectrecognitionanddetection AT yuewang directtraininghighperformancespikingneuralnetworksforobjectrecognitionanddetection AT yuzhang directtraininghighperformancespikingneuralnetworksforobjectrecognitionanddetection AT yuzhang directtraininghighperformancespikingneuralnetworksforobjectrecognitionanddetection |