Kanerva++: Extending the Kanerva Machine With Differentiable, Locally Block Allocated Latent Memory
Mai 3, 2021
Sprecher:innen
Jason Ramapuram
Sprecher:in · 0 Follower:innen
Alexandros Kalousis
Sprecher:in · 0 Follower:innen
Über
Episodic and semantic memory are critical components of the human memory model. The theory of complementary learning system (McClelland et al., 1995) suggests that the compressed representation produced by a serial event (episodic memory) is later restructured to build a more generalized form of reusable knowledge (semantic memory). In this work, we develop a new principled Bayesian memory allocation scheme that bridges the gap between episodic and semantic memory via a hierarchical latent variable model. We take inspiration from traditional heap allocation and extend the idea of locally contiguous memory to the Kanerva Machine, enabling a novel differentiable block allocated latent memory. In contrast to the Kanerva Machine, we simplify the process of memory writing by treating it as a fully feed forward deterministic process, relying on the stochasticity of the read key distribution to disperse information within the memory. We demonstrate that this allocation scheme improves performance in conditional image generation, resulting in new state-of-the-art likelihood values on binarized MNIST (≤41.58 nats/image) , binarized Omniglot (≤66.24 nats/image), as well as presenting competitive performance on CIFAR10, DMLab Mazes, Celeb-A and ImageNet32×32.Episodic and semantic memory are critical components of the human memory model. The theory of complementary learning system (McClelland et al., 1995) suggests that the compressed representation produced by a serial event (episodic memory) is later restructured to build a more generalized form of reusable knowledge (semantic memory). In this work, we develop a new principled Bayesian memory allocation scheme that bridges the gap between episodic and semantic memory via a hierarchical latent varia…
Über ICLR 2021
The International Conference on Learning Representations (ICLR) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning, but generally referred to as deep learning. ICLR is globally renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, text understanding, gaming, and robotics.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Deciphering and Optimizing Multi-Task Learning: a Random Matrix Approach
Später ansehen
Regularization Can Help Mitigate Poisoning Attacks... with the Right Hyperparameters
Später ansehen
Learning structure vs how to use it: Meta-learning inference strategies
Später ansehen
AI-Assisted Tracking of Non-pharmaceutical Interventions Implemented Worldwide for COVID-19
Später ansehen
Tent: Fully Test-Time Adaptation by Entropy Minimization
Später ansehen
Dequan Wang, …
