Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Exploring Example Influence in Continual Learning
Authors: Qing Sun, Fan Lyu, Fanhua Shang, Wei Feng, Liang Wan
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results show that our algorithm significantly outperforms state-of-the-art methods on both taskand class-incremental benchmark CL datasets. |
| Researcher Affiliation | Academia | Qing Sun , Fan Lyu , Fanhua Shang, Wei Feng, Liang Wan College of Intelligence and Computing, Tianjin University EMAIL |
| Pseudocode | Yes | Algorithm 1: Computation of Example Influence (Meta SP) Input: Bold, Bnew, Vold, Vnew ; // Training batches, Validation batches Output: I ; // Pareto example influence on SP ... Algorithm 2: Using Example Influence in Rehearsal-based Continual Learning. ... Algorithm 3: Training New Task |
| Open Source Code | Yes | https://github.com/SSSun Qing/Example_Influence_CL |
| Open Datasets | Yes | We use three commonly used benchmarks for evaluation: 1) Split CIFAR-10 [37]... 2) Split CIFAR-100 [37]... 3) Split Mini-Imagenet [35] is a subset of 100 classes from Image Net [9], rescaled to 32 32. Each class has 600 samples, randomly subdivided into training (80%) and test sets (20%). |
| Dataset Splits | Yes | Split Mini-Imagenet is a subset of 100 classes from Image Net [9], rescaled to 32 32. Each class has 600 samples, randomly subdivided into training (80%) and test sets (20%). We construct the SP validation sets in Meta SP by randomly sampling 10% of the seen data and 10% of the memory buffer at each training step. |
| Hardware Specification | No | No specific hardware details such as GPU/CPU models or processor types were provided in the paper text. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions) were mentioned in the paper. |
| Experiment Setup | Yes | We employ Res Net-18 [17] as the backbone which is trained from scratch. We use Stochastic Gradient Descent (SGD) optimizer and set the batch size 32 unchanged in order to guarantee an equal number of updates. Also, the rehearsal batch sampled from memory buffer is set to 32. We construct the SP validation sets in Meta SP by randomly sampling 10% of the seen data and 10% of the memory buffer at each training step. We set other hyper-settings following ER tricks [4], including 50 total epochs and hyper-parameters. |