Learning rule influences recurrent network representations but not attractor structure in decision-making tasks
Dec 6, 2021
Speakers
Brandon J. McMahan
Speaker · 0 followers
Michael Kleinman
Speaker · 0 followers
Jonathan C. Kao
Speaker · 0 followers
About
Recurrent neural networks (RNNs) are popular tools for studying computational dynamics in neurobiological circuits. However, due to the dizzying array of design choices, it is unclear if computational dynamics unearthed from RNNs provide reliable neurobiological inferences. Addressing these questions is valuable in two ways. First, identification of invariant properties that persist in RNNs across a wide range of design choices are more likely to be candidate neurobiological mechanisms. Second, understanding the effect of design choices on RNN training provides insights into interpretability. We focus our investigation on how RNN learning rule and task design affect RNN computation. We trained large populations of RNNs with different, but commonly used, learning rules on decision-making tasks inspired by neuroscience literature. We find that attractor topology is invariant to the choice of learning rule, but representational geometry is not for more complex tasks. For simple tasks, we find that attractor topology depends on task input noise. However, when a task becomes increasingly complex, RNN attractor topology becomes invariant to input noise. Together, our results suggest that RNN dynamics are robust across learning rules but can be sensitive to the training task design, especially for simpler tasks.Recurrent neural networks (RNNs) are popular tools for studying computational dynamics in neurobiological circuits. However, due to the dizzying array of design choices, it is unclear if computational dynamics unearthed from RNNs provide reliable neurobiological inferences. Addressing these questions is valuable in two ways. First, identification of invariant properties that persist in RNNs across a wide range of design choices are more likely to be candidate neurobiological mechanisms. Second,…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Spectral embedding for dynamic networks with stability guarantees
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learnability of Linear Thresholds from Label Proportions
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Distributed Zero-Order Optimization under Adversarial Noise
Watch later
Arya Akhavan, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Cooperative Multi-Agent Reinforcement Learning for High-Dimensional Nonequilibrium Control
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learning to Generate Visual Questions with Noisy Supervision
Watch later
Kai Shen, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Opening Remarks to Session 1
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%