Learning rule influences recurrent network representations but not attractor structure in decision-making tasks
Dec 6, 2021
Speakers
Brandon J. McMahan
Speaker · 0 followers
Michael Kleinman
Speaker · 0 followers
Jonathan C. Kao
Speaker · 0 followers
About
Recurrent neural networks (RNNs) are popular tools for studying computational dynamics in neurobiological circuits. However, due to the dizzying array of design choices, it is unclear if computational dynamics unearthed from RNNs provide reliable neurobiological inferences. Addressing these questions is valuable in two ways. First, identification of invariant properties that persist in RNNs across a wide range of design choices are more likely to be candidate neurobiological mechanisms. Second, understanding the effect of design choices on RNN training provides insights into interpretability. We focus our investigation on how RNN learning rule and task design affect RNN computation. We trained large populations of RNNs with different, but commonly used, learning rules on decision-making tasks inspired by neuroscience literature. We find that attractor topology is invariant to the choice of learning rule, but representational geometry is not for more complex tasks. For simple tasks, we find that attractor topology depends on task input noise. However, when a task becomes increasingly complex, RNN attractor topology becomes invariant to input noise. Together, our results suggest that RNN dynamics are robust across learning rules but can be sensitive to the training task design, especially for simpler tasks.Recurrent neural networks (RNNs) are popular tools for studying computational dynamics in neurobiological circuits. However, due to the dizzying array of design choices, it is unclear if computational dynamics unearthed from RNNs provide reliable neurobiological inferences. Addressing these questions is valuable in two ways. First, identification of invariant properties that persist in RNNs across a wide range of design choices are more likely to be candidate neurobiological mechanisms. Second,…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Searching Parameterized AP Loss for Object Detection
Watch later
Chenxin Tao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learning Large Neighborhood Search Policy for Integer Programming
Watch later
Yaoxin Wu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Panel Discussion 2
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Simple steps are all you need: Frank-Wolfe and generalized self-concordant functions
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data
Watch later
Feng Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Reinforcement Learning in Linear MDPs: Constant Regret and Representation Selection
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%