DualNet: Continual Learning, Fast and Slow
Dec 6, 2021
Speakers
Quang Pham
Speaker · 0 followers
Cheng-Hao Liu
Speaker · 0 followers
Steven Hoi
Speaker · 0 followers
About
According to Complementary Learning Systems (CLS) theory <cit.> in neuroscience, humans do effective continual learning through two complementary systems: a fast learning system centered on the hippocampus for rapid learning of the specifics and individual experiences, and a slow learning system located in the neocortex for the gradual acquisition of structured knowledge about the environment. Motivated by this theory, we propose a novel continual learning framework named “DualNet", which comprises a fast learning system for supervised learning of pattern-separated representation from specific tasks and a slow learning system for unsupervised representation learning of task-agnostic general representation via a Self-Supervised Learning (SSL) technique. The two fast and slow learning systems are complementary and work seamlessly in a holistic continual learning framework. Our extensive experiments on two challenging continual learning benchmarks of CORE50 and miniImageNet show that DualNet outperforms state-of-the-art continual learning methods by a large margin. We further conduct ablation studies of different SSL objectives to validate DualNet's efficacy, robustness, and scalability. Code will be made available upon acceptance.According to Complementary Learning Systems (CLS) theory <cit.> in neuroscience, humans do effective continual learning through two complementary systems: a fast learning system centered on the hippocampus for rapid learning of the specifics and individual experiences, and a slow learning system located in the neocortex for the gradual acquisition of structured knowledge about the environment. Motivated by this theory, we propose a novel continual learning framework named “DualNet", …
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Demand Prediction of Mobile Clinics using Public Data
Watch later
Haipeng Chen, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Fast Tucker Rank Reduction for Non-Negative Tensors Using Mean-Field Approximation
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Sifting through the noise: Universal first-order methods for stochastic variational inequalities
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Representer Point Selection via Local Jacobian Expansion for Classifier Explanation of Deep Neural Networks and Ensemble Models
Watch later
Yi Sui, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
ML in urban planning panel
Watch later
Aditya Mate, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Directional Message Passing on Molecular Graphs via Synthetic Coordinates
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%