Sparse Spiking Gradient Descent
Dec 6, 2021
Speakers
Nicolas Perez-Nieves
Speaker · 0 followers
Dan F. M. Goodman
Speaker · 0 followers
About
There is an increasing interest in emulating Spiking Neural Networks (SNNs) on neuromorphic computing devices due to their low energy consumption. Recent advances have allowed training SNNs to a point where they start to compete with traditional Artificial Neural Networks (ANNs) in terms of accuracy, while at the same time being energy efficient when run on neuromorphic hardware. However, the process of training SNNs is still based on dense tensor operations originally developed for ANNs which do not leverage the spatiotemporally sparse nature of SNNs. We present here the first sparse SNN backpropagation algorithm which achieves the same or better accuracy as current state of the art methods while being significantly faster and more memory efficient. We show the effectiveness of our method on real datasets of varying complexity (Fashion-MNIST, Neuromophic-MNIST and Spiking Heidelberg Digits) achieving a speedup in the backward pass of up to 70x, and 40% more memory efficient, without losing accuracy.There is an increasing interest in emulating Spiking Neural Networks (SNNs) on neuromorphic computing devices due to their low energy consumption. Recent advances have allowed training SNNs to a point where they start to compete with traditional Artificial Neural Networks (ANNs) in terms of accuracy, while at the same time being energy efficient when run on neuromorphic hardware. However, the process of training SNNs is still based on dense tensor operations originally developed for ANNs which d…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Structural Credit Assignment in Neural Networks using Reinforcement Learning
Watch later
Dhawal Gupta, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Data Cards: Purposeful and Transparent Documentation for Responsible AI
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
NovelD: A Simple yet Effective Exploration Criterion
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development
Watch later
Kexin Huang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Inverse Reinforcement Learning in a Continuous State Space with Formal Guarantees
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Stochastic Shortest Path: Minimax, Parameter-Free and Towards Horizon-Free Regret
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%