Enhancing Knowledge Transfer for Task Incremental Learning with Data-free Subnetwork
Authors: Qiang Gao, Xiaojun Shan, Yuchen Zhang, Fan Zhou
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The comprehensive experiments conducted on four benchmark datasets demonstrate the effectiveness of the proposed DSN in the context of task-incremental learning by comparing it to several state-of-the-art baselines. |
| Researcher Affiliation | Academia | 1Southwestern University of Finance and Economics, Chengdu, China 2University of Electronic Science and Technology of China, Chengdu, China |
| Pseudocode | Yes | Algorithm 1: Incremental learning with DSN. |
| Open Source Code | Yes | The source codes are available at https://github.com/shanxiaojun/DSN. |
| Open Datasets | Yes | We employ four benchmark datasets for TIL problem as follows: Permuted MNIST (PMNIST) [3], Rotated MNIST (RMNIST) [21], Incremental CIFAR-100 [54, 55], and Tiny Image Net [19]. PMNIST encompasses 10 variations of MNIST [56]... The original CIFAR-100 was divided into 20 tasks... Tiny Image Net constitutes a variant of Image Net [57]... |
| Dataset Splits | No | The paper mentions 'test sets' but does not explicitly provide details about train/validation/test dataset splits, such as percentages, sample counts, or a specific splitting methodology in the main text. |
| Hardware Specification | No | The paper does not specify any particular hardware components like CPU or GPU models used for experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | Yes | For two variants of MNIST, we adopt the experimental setup outlined in [44] in which we use a two-layer MLP, a multi-head classifier, and begin with 2000-2000-10 neurons for the first task. ...We evaluate the impact of η that aims to hold more room for future tasks. ... η = 0 indicates that we do not penalize the number of activated neurons for each task. |