Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion
The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learni...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/24/4/455 |
| _version_ | 1797434800022224896 |
|---|---|
| author | Shuangming Yang Jiangtong Tan Badong Chen |
| author_facet | Shuangming Yang Jiangtong Tan Badong Chen |
| author_sort | Shuangming Yang |
| collection | DOAJ |
| description | The spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems. |
| first_indexed | 2024-03-09T10:37:59Z |
| format | Article |
| id | doaj.art-b6b33431e28547d89e3942b727429163 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-09T10:37:59Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-b6b33431e28547d89e3942b7274291632023-12-01T20:49:55ZengMDPI AGEntropy1099-43002022-03-0124445510.3390/e24040455Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy CriterionShuangming Yang0Jiangtong Tan1Badong Chen2School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, ChinaSchool of Electrical and Information Engineering, Tianjin University, Tianjin 300072, ChinaInstitute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an 710049, ChinaThe spiking neural network (SNN) is regarded as a promising candidate to deal with the great challenges presented by current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the online meta-learning performance of artificial neural networks. Importantly, existing spike-based online meta-learning models do not target the robust learning based on spatio-temporal dynamics and superior machine learning theory. In this invited article, we propose a novel spike-based framework with minimum error entropy, called MeMEE, using the entropy theory to establish the gradient-based online meta-learning scheme in a recurrent SNN architecture. We examine the performance based on various types of tasks, including autonomous navigation and the working memory test. The experimental results show that the proposed MeMEE model can effectively improve the accuracy and the robustness of the spike-based meta-learning performance. More importantly, the proposed MeMEE model emphasizes the application of the modern information theoretic learning approach on the state-of-the-art spike-based learning algorithms. Therefore, in this invited paper, we provide new perspectives for further integration of advanced information theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems.https://www.mdpi.com/1099-4300/24/4/455spiking neural networkmeta-learninginformation theoretic learningminimum error entropyartificial general intelligence |
| spellingShingle | Shuangming Yang Jiangtong Tan Badong Chen Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion Entropy spiking neural network meta-learning information theoretic learning minimum error entropy artificial general intelligence |
| title | Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion |
| title_full | Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion |
| title_fullStr | Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion |
| title_full_unstemmed | Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion |
| title_short | Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion |
| title_sort | robust spike based continual meta learning improved by restricted minimum error entropy criterion |
| topic | spiking neural network meta-learning information theoretic learning minimum error entropy artificial general intelligence |
| url | https://www.mdpi.com/1099-4300/24/4/455 |
| work_keys_str_mv | AT shuangmingyang robustspikebasedcontinualmetalearningimprovedbyrestrictedminimumerrorentropycriterion AT jiangtongtan robustspikebasedcontinualmetalearningimprovedbyrestrictedminimumerrorentropycriterion AT badongchen robustspikebasedcontinualmetalearningimprovedbyrestrictedminimumerrorentropycriterion |