Model Sensitivity Aware Continual Learning
Authors: Zhenyi Wang, Heng Huang
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments conducted across multiple datasets demonstrate the efficacy and versatility of our proposed method. |
| Researcher Affiliation | Academia | Zhenyi Wang and Heng Huang Department of Computer Science, Institute of Health Computing University of Maryland College Park College Park, MD, 20742 zwang@umd.edu;heng@umd.edu |
| Pseudocode | Yes | Algorithm 1 Model Sensitivity Aware Continual Learning |
| Open Source Code | No | Justification: Code will be released. |
| Open Datasets | Yes | We conduct experiments on several datasets, including CIFAR10 (10 classes), CIFAR100 (100 classes) [29], and Tiny-Image Net (200 classes) [80], to assess the effectiveness of MACL in task incremental learning (Task-IL) and class incremental learning (Class-IL). |
| Dataset Splits | No | Following [7, 14], the hyperparameter is determined through the validation sets split from the training sets from the first three tasks. |
| Hardware Specification | Yes | We use a single NVIDIA A5000 GPU with 24GB memory to run the experiments. |
| Software Dependencies | No | The paper mentions using 'standard SGD optimizer' but does not specify versions for programming languages, libraries (e.g., PyTorch, TensorFlow), or other software dependencies. |
| Experiment Setup | Yes | For the hyperparameters in our method, we set α = 1.0 across all the datasets to minimize the model’s dependence on hyperparameters. For η, we set η = 1e 5 for CIFAR10 and CIFAR100, and η = 1e 6 for Tiny-Image Net. The η is selected from the range of [1e 4, 1e 5, 1e 6, 1e 7]. ... The batch size and replay buffer batch size are set to 32. |