Neural Wave Machines
Jul 24, 2023
Speakers
T. Anderson Keller
Speaker · 0 followers
Max Welling
Speaker · 31 followers
About
Traveling waves have been measured at a diversity of regions and scales in the brain, however a consensus as to their computational purpose has yet to be reached. An intriguing hypothesis is that traveling waves serve to structure neural representations both in space and time, thereby acting as an inductive bias towards natural data. In this work, we investigate this hypothesis by introducing the Neural Wave Machine (NWM) – a locally coupled oscillatory recurrent neural network capable of exhibiting traveling waves in its hidden state. After training on simple dynamic sequences, we show that this model indeed learns static spatial structure such topographic organization, and further uses complex spatiotemporal structure such traveling waves to encode observed transformations. To measure the computational implications of this structure, we use a suite of sequence classification and physical dynamics modeling tasks to show that the NWM is both more parameter efficient, and is able to forecast future trajectories of simple physical dynamical systems more accurately than existing state of the art counterparts.Traveling waves have been measured at a diversity of regions and scales in the brain, however a consensus as to their computational purpose has yet to be reached. An intriguing hypothesis is that traveling waves serve to structure neural representations both in space and time, thereby acting as an inductive bias towards natural data. In this work, we investigate this hypothesis by introducing the Neural Wave Machine (NWM) – a locally coupled oscillatory recurrent neural network capable of exhibi…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Protecting the future: Data sovereignty and the Kaitiakitanga License
Watch later
Repurposing Density Functional Theory to Suit Deep Learning
Watch later
Training Spiking Neural Networks with Local Tandem Learning
Watch later
Learning Fair Representations
Watch later
On Excess Mass Behavior in Gaussian Mixture Models with Orlicz-Wasserstein Distances
Watch later
Aritra Guha, …
ClusterFuG: Clustering Fully connected Graphs by Multicut
Watch later
Ahmed Abbas, …