DualHSIC: HSIC-Bottleneck and Alignment for Continual Learning
Authors: Zifeng Wang, Zheng Zhan, Yifan Gong, Yucai Shao, Stratis Ioannidis, Yanzhi Wang, Jennifer Dy
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To evaluate the efficacy of the proposed Dual HSIC, we conduct comprehensive experiments on representative CL benchmarks, closely following the challenging class-incremental learning setting in prior works (Lopez-Paz & Ranzato, 2017; Van de Ven & Tolias, 2019; Wang et al., 2022c). We incorporate Dual HSIC with multiple SOTA rehearsal-based CL methods to demonstrate performance improvement, while also comparing Dual HSIC against other SOTA CL methods. We also performed an ablation study and exploratory experiments to further showcase the effectiveness of individual components. |
| Researcher Affiliation | Academia | 1Northeastern University 2University of California, Los Angeles. |
| Pseudocode | Yes | Algorithm 1 Dual HSIC for Continual Learning |
| Open Source Code | Yes | Our code is publicly available1. 1https://github.com/zhanzheng8585/Dual HSIC |
| Open Datasets | Yes | Split CIFAR-10 originates from the well-known CIFAR10 (Krizhevsky et al., 2009) dataset. [...] Split CIFAR-100 is also a split version of CIFAR100 (Krizhevsky et al., 2009) [...]. Split mini Image Net is subsampled from Image Net (Deng et al., 2009) with 100 classes. |
| Dataset Splits | Yes | Specifically, we subsample the training set by a ratio of 20% in a stratified way for every task before the continual learning process starts. The subsampled data are put aside as a validation set. |
| Hardware Specification | Yes | All experiments are conducted on a single Tesla V100 GPU with 32GB memory. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies like programming languages or libraries (e.g., PyTorch, TensorFlow, Python version, CUDA version). |
| Experiment Setup | Yes | The pertask training epochs are set as 50 for Split CIFAR-10/100, and 80 for Split mini Image Net. The batch sizes are set as 32, 64, and 128 for Split CIFAR-10, Split CIFAR-100, and Split mini Image Net, respectively. |