Abstract:
Experience replay (ER) is a widely-adopted neuroscience-inspired method to perform lifelong learning. Nonetheless, existing ER-based approaches consider very coarse memory modules with simple memory and rehearsal mechanisms that cannot fully exploit the potential of memory replay. Evidence from neuroscience has provided fine-grained memory and rehearsal mechanisms, such as the dual-store memory system consisting of PFC-HC circuits. However, the computational abstraction of these processes is still very challenging. To address these problems, we introduce the Dual-Memory (Dual-MEM) model emulating the memorization, consolidation, and rehearsal process in the PFC-HC dual-store memory circuit. Dual-MEM maintains an incrementally updated short-term memory to benefit current-task learning. At the end of the current task, short-term memories will be consolidated into long-term ones for future rehearsal to alleviate forgetting. For the Dual-MEM optimization, we propose two learning policies that emulate different memory retrieval strategies: Direct Retrieval Learning and Mixup Retrieval Learning. Extensive evaluations on eight benchmarks demonstrate that Dual-MEM delivers compelling performance while maintaining high learning and memory utilization efficiencies under the challenging experience-once setting.
摘要:
经验重播(ER)是一种广泛采用的神经科学启发方法来进行终身学习。尽管如此,现有的基于ER的方法考虑具有简单存储器和演练机制的非常粗糙的存储器模块,其不能完全利用存储器重放的潜力。神经科学的证据提供了细粒度的记忆和排练机制,例如由PFC-HC电路组成的双存储存储器系统。然而,这些过程的计算抽象仍然非常具有挑战性。为了解决这些问题,我们介绍了模拟记忆的双记忆(Dual-MEM)模型,合并,PFC-HC双存储存储器电路中的演练过程。双MEM保持增量更新的短期记忆,以有利于当前任务学习。在当前任务结束时,短期记忆将合并为长期记忆,以备将来排练,以减轻遗忘。对于双MEM优化,我们提出了两种学习策略来模拟不同的记忆检索策略:直接检索学习和混合检索学习。对八个基准的广泛评估表明,Dual-MEM在一次具有挑战性的体验设置下,可提供引人注目的性能,同时保持较高的学习和记忆利用效率。