Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding
Authors: Depeng Li, Tianqi Wang, Junwei Chen, Qining Ren, Kenji Kawaguchi, Zhigang Zeng
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02 the base model. [...] Empirical evaluation across a range of widely used benchmark datasets demonstrates the superiority of our approach in terms of exemplar buffers, network expansion, and competitive performance. [...] We perform extensive experiments to evaluate the proposed CLDNet in the challenging class-IL setting. |
| Researcher Affiliation | Academia | 1School of Artificial Intelligence and Automation, Huazhong University of Science and Technology 2School of Computing, National University of Singapore |
| Pseudocode | Yes | Algorithm 1: CLDNet Training and Test algorithm |
| Open Source Code | No | The paper does not explicitly state that source code is released or provide a link to a code repository. |
| Open Datasets | Yes | Small Scale: MNIST (Le Cun et al. 1998) contains 60,000 handwritten digit images in the training set and 10,000 samples in the test set... Fashion MNIST (Xiao, Rasul, and Vollgraf 2017)... CIFAR-10 (Krizhevsky, Hinton et al. 2009)... Medium Scale: CIFAR-100 (Krizhevsky, Hinton et al. 2009)... Large Scale: Image Net-R (Hendrycks et al. 2021)... |
| Dataset Splits | No | The paper specifies overall training and testing set sizes and how classes are divided into tasks but does not explicitly detail a validation split or its size/percentage for reproducibility. |
| Hardware Specification | Yes | All experiments are run in Py Torch using NVIDIA RTX 3080-Ti GPUs with 12GB memory. |
| Software Dependencies | No | The paper mentions 'Py Torch' but does not specify its version or the versions of other ancillary software components used for reproducibility. |
| Experiment Setup | Yes | In our CLDNet, for HBO we set the coefficient β = 500 and adopt the Gaussian kernel as suggested by (Ma, Lewis, and Kleijn 2020), as well as the adaptive α with an initial value 0.01 for the orthogonal projector, like (Guo et al. 2022); For EAE we set γ = 0.04, d = 1000, and C = 1000 following the recommendations by EBVs (Shen, Sun, and Wei 2023). |