Improving Task-free Continual Learning by Distributionally Robust Memory Evolution
Authors: Zhenyi Wang, Li Shen, Le Fang, Qiuling Suo, Tiehang Duan, Mingchen Gao
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on existing benchmarks demonstrate the effectiveness of the proposed methods for alleviating forgetting. As a by-product of the proposed framework, our method is more robust to adversarial examples than existing task-free CL methods. |
| Researcher Affiliation | Collaboration | Zhenyi Wang 1 Li Shen 2 Le Fang 1 Qiuling Suo 1 Tiehang Duan 3 Mingchen Gao 1 1Department of Computer Science and Engineering, University at Buffalo, NY, USA 2JD Explore Academy, Beijing, China 3Meta, Seattle, WA, USA. |
| Pseudocode | Yes | Algorithm 1 Distributionally Robust Memory Evolution. |
| Open Source Code | No | The paper does not provide an explicit statement or link for open-source code for its methodology. |
| Open Datasets | Yes | CIFAR10, following (Aljundi et al., 2019a), we split the CIFAR-10 dataset into 5 disjoint tasks with the same training, validation, and test sets. Mini Imagenet, following (Aljundi et al., 2019a), we split Mini Imagenet (Vinyals et al., 2016) dataset with 100 image classes into 20 disjoint tasks. Each task has 5 classes. CIFAR-100 (Krizhevsky, 2009), contains 100 image classes. We also split it into 20 disjoint tasks. |
| Dataset Splits | Yes | we split the CIFAR-10 dataset into 5 disjoint tasks with the same training, validation, and test sets. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware used (e.g., GPU/CPU models, memory details) for running its experiments. |
| Software Dependencies | No | The paper mentions using Resnet-18 but does not provide specific software dependencies with version numbers (e.g., Python version, deep learning framework versions like PyTorch or TensorFlow, or library versions). |
| Experiment Setup | Yes | Implementation Details. We use the Resnet-18 as (Aljundi et al., 2019a). The number of evolution steps T is set to be 5, the evolution rate α is set to be 0.01 for CIFAR10, 0.05 for CIFAR100 and 0.001 for Mini Imagenet, β = 0.003 and momentum τ = 0.1. We set the memory buffer size to be 500 for CIFAR-10 dataset, 5K for CIFAR-100 and 10K for Mini Imagenet. All other hyperparameters are the same as (Aljundi et al., 2019a). All reported results in our experiments are the average accuracy of 10 runs with standard deviation. |