Sparse Coding in a Dual Memory System for Lifelong Learning
Authors: Fahad Sarfraz, Elahe Arani, Bahram Zonooz
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our extensive evaluation and characteristics analysis show that equipped with these biologically inspired mechanisms, the model can further mitigate forgetting. Code available at https://github.com/Neur AI-Lab/SCo MMER. Our empirical evaluation on challenging CL settings and characteristic analysis show that equipping the model with these biologically inspired mechanisms can further mitigate forgetting and effectively consolidate information across the tasks. Table 1: Comparison on different CL settings. The baseline results are from (Arani, Sarfraz, and Zonooz 2022). Table 2: Ablation Study: Effect of systematically removing different components of SCo MMER on the performance in S-CIFAR10. |
| Researcher Affiliation | Collaboration | 1Advanced Research Lab, Nav Info Europe, The Netherlands 2Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlands fahad.sarfraz@navinfo.eu, elahe.arani@tue.nl, bahram.zonooz@gmail.com |
| Pseudocode | Yes | Algorithm 1: SCo MMER Algorithm for Sparse Coding in Multiple Memory Experience Replay System |
| Open Source Code | Yes | Code available at https://github.com/Neur AI-Lab/SCo MMER. |
| Open Datasets | Yes | We compare SCo MMER with state-of-the-art rehearsal-based methods across different CL settings under uniform experimental settings (details provided in Appendix). SGD provides the lower bound with standard training on sequential tasks, and JOINT gives the upper bound on performance when the model is trained on the joint distribution. Table 1 shows that SCo MMER provides performance gains in the majority of the cases and demonstrates the effectiveness of our approach under varying challenging CL settings. In particular, it provides considerable improvement under low buffer size settings, which suggests that our method is able to mitigate forgetting with fewer samples from previous tasks. The paper refers to 'S-CIFAR10' and 'S-CIFAR100' in Table 1, which are widely recognized public datasets, albeit with an 'S-' prefix likely indicating a specific variant or split. |
| Dataset Splits | No | No specific training/validation/test dataset splits (e.g., percentages, sample counts, or explicit references to standard splits) are provided in the main text. The paper states 'Details of the datasets used in each setting are provided in Appendix,' but the Appendix is not supplied. |
| Hardware Specification | No | No specific hardware details such as GPU models (e.g., NVIDIA A100, Tesla V100), CPU models, or memory specifications are mentioned for running the experiments. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., 'PyTorch 1.9', 'Python 3.8') are mentioned in the paper. |
| Experiment Setup | No | No specific experimental setup details, such as hyperparameter values (e.g., learning rate, batch size, number of epochs) or specific optimizer settings, are provided in the main text. The paper states 'details provided in Appendix,' but the Appendix is not supplied. |