NeurIPS2021-DualNet: Continual Learning, Fast and Slow

Abstract

神经科学的理论complementary learning systems; fast学当前具体任务信息（supervised），slow学通用的(self-supervised)

Introduction

the main focus of this study is exploring how the CLS theory can motivate a general continual learning framework with a better trade-off between alleviating catastrophic forgetting and facilitating knowledge transfer.
A single backbone to model both the hippocampus and neocortex; Figure1, slow learner use samples from the memory;

框架上来看，universal里面slow net利用自监督学习，通过memory的样本训练；
Focus在online continual learning, 传统的batch continual learning,

Method

Online continual learning setting.
区别task-aware setting, where only the correspon