SparCL: Sparse Continual Learning on the Edge
Authors: Zifeng Wang, Zheng Zhan, Yifan Gong, Geng Yuan, Wei Niu, Tong Jian, Bin Ren, Stratis Ioannidis, Yanzhi Wang, Jennifer Dy
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct comprehensive experiments on multiple CL benchmarks to evaluate the effectiveness of our method. We show that Spar CL works collaboratively with existing CL methods, greatly accelerates the learning process under different sparsity ratios, and even sometimes improves upon the state-of-the-art accuracy. We also establish competing baselines by combining representative sparse training methods with advanced rehearsal-based CL methods. Spar CL again outperforms these baselines in terms of both efficiency and accuracy. Most importantly, we evaluate our Spar CL framework on real edge devices to demonstrate the practical potential of our method. |
| Researcher Affiliation | Academia | Zifeng Wang1, , Zheng Zhan1, , Yifan Gong1, Geng Yuan1, Wei Niu2, Tong Jian1, Bin Ren2, Stratis Ioannidis1, Yanzhi Wang1, Jennifer Dy1 1 Northeastern University, 2 College of William and Mary {zhan.zhe, gong.yifa, geng.yuan, yanz.wang}@northeastern.edu, {zifengwang, jian, ioannidis, jdy}@ece.neu.edu, wniu@email.wm.edu, bren@cs.wm.edu |
| Pseudocode | Yes | Algorithm 1: Task-aware Dynamic Masking (TDM) |
| Open Source Code | Yes | Our code is publicly available . https://github.com/neu-spiral/Spar CL |
| Open Datasets | Yes | We evaluate our Spar CL on two representative CL benchmarks, Split CIFAR-10 [33] and Split Tiny-Image Net [16] to verify the efficacy of Spar CL. ... Dataset licensing information can be found in Appendix A. |
| Dataset Splits | No | The paper mentions using Split CIFAR-10 and Split Tiny-Image Net and discusses training and testing, but it does not explicitly specify the percentages or counts for training, validation, and test dataset splits. |
| Hardware Specification | Yes | The training acceleration results are measured on the CPU of an off-the-shelf Samsung Galaxy S20 smartphone, which has the Qualcomm Snapdragon 865 mobile platform with a Qualcomm Kryo 585 Octa-core CPU. |
| Software Dependencies | No | The paper describes the models and methods used (e.g., Res Net-18, DER++, ER), but it does not specify software dependencies with version numbers such as Python, PyTorch, or TensorFlow. |
| Experiment Setup | Yes | We set the per task training epochs to 50 and 100 for Split CIFAR-10 and Tiny-Image Net, respectively, with a batch size of 32. For the model architecture, we follow [8, 50] and adopt the Res Net-18 [26] without any pre-training. We also use the best hyperparameter setting reported in [8, 57] for CL methods, and in [20, 35] for CL-adapted sparse training methods. |