Continual Learning with Guarantees via Weight Interval Constraints
Authors: Maciej Wołczyk, Karol Piczak, Bartosz Wójcik, Lukasz Pustelnik, Paweł Morawiecki, Jacek Tabor, Tomasz Trzcinski, Przemysław Spurek
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our claim by developing Inter Conti Net (Interval Continual Learning) algorithm which leverages interval arithmetic to effectively model parameter regions as hyperrectangles. Through experimental results, we show that our approach performs well in a continual learning setup without storing data from previous tasks. (...) 5. Experiments To verify the empirical usefulness of our method, we test it in three standard continual learning scenarios... |
| Researcher Affiliation | Collaboration | 1Faculty of Mathematics and Computer Science, Jagiellonian University, Krak ow, Poland 2Institute of Computer Science, Polish Academy of Sciences, Warsaw, Poland 3Warsaw University of Technology, Warsaw, Poland 4Tooploox. |
| Pseudocode | Yes | Algorithm 1 Inter Conti Net training procedure for a given task |
| Open Source Code | Yes | The code is available at https://github.com/gmum/ Inter Conti Net. |
| Open Datasets | Yes | We run the experiments on four datasets commonly used for continual learning: MNIST, Fashion MNIST, CIFAR-10 and CIFAR-100 split into sequences of tasks. (...) Data We use MNIST, Fashion MNIST, CIFAR-10, and CIFAR-100 datasets with original training and testing splits. |
| Dataset Splits | Yes | Data We use MNIST, Fashion MNIST, CIFAR-10, and CIFAR-100 datasets with original training and testing splits. We split the datasets into 5 tasks containing data from classes: [0, 1], [2, 3], [4, 5], [6, 7], [8, 9]. We split CIFAR-100 into 20 tasks with 5 classes. |
| Hardware Specification | No | The paper mentions training on 'GPUs' but does not provide specific hardware details such as GPU model numbers, CPU specifications, or memory amounts. |
| Software Dependencies | No | The paper mentions using 'Avalanche' library and 'Py Torch' framework but does not specify their version numbers. |
| Experiment Setup | Yes | In the MNIST and Fashion-MNIST experiments we use batch size of 128, 30 epochs of training for each task (...) The learning rate is set to 0.001 for MNIST and Fashion MNIST experiments, and 0.01 for split-CIFAR-10 and split-CIFAR-100. (...) Table 7. Hyperparameters for Inter Conti Net on each dataset and setting combination. |