GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks
Nov 28, 2022
Speakers
Xingting Yao
Speaker · 0 followers
Fanrong Li
Speaker · 0 followers
Jian Cheng
Speaker · 1 follower
About
Spiking Neural Networks (SNNs) have been studied over decades to incorporate their biological plausibility and leverage their promising energy efficiency. Throughout existing SNNs, the leaky integrate-and-fire (LIF) model is commonly adopted to formulate the spiking neuron and evolves into numerous variants with different biological features. However, most LIF-based neurons support only single biological feature in different neuronal behaviors, limiting their expressiveness and neuronal dynamic diversity. In this paper, we propose GLIF, a unified spiking neuron, to fuse different bio-features in different neuronal behaviors, enlarging the representation space of spiking neurons. In GLIF, gating factors, which are exploited to determine the proportion of the fused bio-features, are learnable during training. Combining all learnable membrane-related parameters, our method can make spiking neurons different and constantly changing, thus increasing the heterogeneity and adaptivity of spiking neurons. Extensive experiments on a variety of datasets demonstrate that our method obtains superior performance compared with other SNNs by simply changing their neuronal formulations to GLIF. In particular, we train a spiking ResNet-19 with GLIF and achieve 77.35% top-1 accuracy with six time steps on CIFAR-100, which has advanced the state-of-the-art.Spiking Neural Networks (SNNs) have been studied over decades to incorporate their biological plausibility and leverage their promising energy efficiency. Throughout existing SNNs, the leaky integrate-and-fire (LIF) model is commonly adopted to formulate the spiking neuron and evolves into numerous variants with different biological features. However, most LIF-based neurons support only single biological feature in different neuronal behaviors, limiting their expressiveness and neuronal dynamic…
Organizer
NeurIPS 2022
Account · 956 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Generalised Implicit Neural Representations
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A Future for AI Governance Systems beyond Predictions
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Signature kernel methods
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
What's the deal with AI safety? Motivations and ways to get involved
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Gradient dynamics of single-neuron autoencoders on orthogonal data
Watch later
Nikhil Ghosh, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%