Learning rule influences recurrent network representations but not attractor structure in decision-making tasks
Dec 6, 2021
Speakers
Brandon J. McMahan
Speaker · 0 followers
Michael Kleinman
Speaker · 0 followers
Jonathan C. Kao
Speaker · 0 followers
About
Recurrent neural networks (RNNs) are popular tools for studying computational dynamics in neurobiological circuits. However, due to the dizzying array of design choices, it is unclear if computational dynamics unearthed from RNNs provide reliable neurobiological inferences. Addressing these questions is valuable in two ways. First, identification of invariant properties that persist in RNNs across a wide range of design choices are more likely to be candidate neurobiological mechanisms. Second, understanding the effect of design choices on RNN training provides insights into interpretability. We focus our investigation on how RNN learning rule and task design affect RNN computation. We trained large populations of RNNs with different, but commonly used, learning rules on decision-making tasks inspired by neuroscience literature. We find that attractor topology is invariant to the choice of learning rule, but representational geometry is not for more complex tasks. For simple tasks, we find that attractor topology depends on task input noise. However, when a task becomes increasingly complex, RNN attractor topology becomes invariant to input noise. Together, our results suggest that RNN dynamics are robust across learning rules but can be sensitive to the training task design, especially for simpler tasks.Recurrent neural networks (RNNs) are popular tools for studying computational dynamics in neurobiological circuits. However, due to the dizzying array of design choices, it is unclear if computational dynamics unearthed from RNNs provide reliable neurobiological inferences. Addressing these questions is valuable in two ways. First, identification of invariant properties that persist in RNNs across a wide range of design choices are more likely to be candidate neurobiological mechanisms. Second,…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Fair Clustering Using Antidote Data
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
CtrlGen: Controllable Generative Modeling in Language and Vision
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Machine Learning in Automating Carbon Sequestration Site Assessment
Watch later
Jay Chen, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Smooth tensor estimation with unknown permutation
Watch later
Chanwoo Lee, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Federated Graph Classification over Non-IID Graphs
Watch later
Han Xie, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A research framework for writing differentiable PDE discretizations in JAX
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%