Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks
Spiking neural networks (SNNs) are recurrent models that can leverage sparsity in input time series to efficiently carry out tasks such as classification. Additional efficiency gains can be obtained if decisions are taken as early as possible as a function of the complexity of the input time series....
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-01-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/26/2/126 |
| _version_ | 1797298352524623872 |
|---|---|
| author | Jiechen Chen Sangwoo Park Osvaldo Simeone |
| author_facet | Jiechen Chen Sangwoo Park Osvaldo Simeone |
| author_sort | Jiechen Chen |
| collection | DOAJ |
| description | Spiking neural networks (SNNs) are recurrent models that can leverage sparsity in input time series to efficiently carry out tasks such as classification. Additional efficiency gains can be obtained if decisions are taken as early as possible as a function of the complexity of the input time series. The decision on when to stop inference and produce a decision must rely on an estimate of the current accuracy of the decision. Prior work demonstrated the use of conformal prediction (CP) as a principled way to quantify uncertainty and support adaptive-latency decisions in SNNs. In this paper, we propose to enhance the uncertainty quantification capabilities of SNNs by implementing ensemble models for the purpose of improving the reliability of stopping decisions. Intuitively, an ensemble of multiple models can decide when to stop more reliably by selecting times at which most models agree that the current accuracy level is sufficient. The proposed method relies on different forms of information pooling from ensemble models and offers theoretical reliability guarantees. We specifically show that variational inference-based ensembles with p-variable pooling significantly reduce the average latency of state-of-the-art methods while maintaining reliability guarantees. |
| first_indexed | 2024-03-07T22:33:39Z |
| format | Article |
| id | doaj.art-79675b522ff24a288add444abe960cfa |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-07T22:33:39Z |
| publishDate | 2024-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-79675b522ff24a288add444abe960cfa2024-02-23T15:15:40ZengMDPI AGEntropy1099-43002024-01-0126212610.3390/e26020126Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural NetworksJiechen Chen0Sangwoo Park1Osvaldo Simeone2KCLIP Laboratory—King’s Communications, Learning and Information Processing Laboratory, Department of Engineering, King’s College London, London WC2R 2LS, UKKCLIP Laboratory—King’s Communications, Learning and Information Processing Laboratory, Department of Engineering, King’s College London, London WC2R 2LS, UKKCLIP Laboratory—King’s Communications, Learning and Information Processing Laboratory, Department of Engineering, King’s College London, London WC2R 2LS, UKSpiking neural networks (SNNs) are recurrent models that can leverage sparsity in input time series to efficiently carry out tasks such as classification. Additional efficiency gains can be obtained if decisions are taken as early as possible as a function of the complexity of the input time series. The decision on when to stop inference and produce a decision must rely on an estimate of the current accuracy of the decision. Prior work demonstrated the use of conformal prediction (CP) as a principled way to quantify uncertainty and support adaptive-latency decisions in SNNs. In this paper, we propose to enhance the uncertainty quantification capabilities of SNNs by implementing ensemble models for the purpose of improving the reliability of stopping decisions. Intuitively, an ensemble of multiple models can decide when to stop more reliably by selecting times at which most models agree that the current accuracy level is sufficient. The proposed method relies on different forms of information pooling from ensemble models and offers theoretical reliability guarantees. We specifically show that variational inference-based ensembles with p-variable pooling significantly reduce the average latency of state-of-the-art methods while maintaining reliability guarantees.https://www.mdpi.com/1099-4300/26/2/126spiking neural networksconformal predictionlatency adaptivityBayesian learning |
| spellingShingle | Jiechen Chen Sangwoo Park Osvaldo Simeone Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks Entropy spiking neural networks conformal prediction latency adaptivity Bayesian learning |
| title | Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks |
| title_full | Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks |
| title_fullStr | Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks |
| title_full_unstemmed | Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks |
| title_short | Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks |
| title_sort | agreeing to stop reliable latency adaptive decision making via ensembles of spiking neural networks |
| topic | spiking neural networks conformal prediction latency adaptivity Bayesian learning |
| url | https://www.mdpi.com/1099-4300/26/2/126 |
| work_keys_str_mv | AT jiechenchen agreeingtostopreliablelatencyadaptivedecisionmakingviaensemblesofspikingneuralnetworks AT sangwoopark agreeingtostopreliablelatencyadaptivedecisionmakingviaensemblesofspikingneuralnetworks AT osvaldosimeone agreeingtostopreliablelatencyadaptivedecisionmakingviaensemblesofspikingneuralnetworks |