Online Boundary-Free Continual Learning by Scheduled Data Prior
Authors: Hyunseo Koh, Minhyuk Seo, Jihwan Bang, Hwanjun Song, Deokki Hong, Seulki Park, Jung-Woo Ha, Jonghyun Choi
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically evaluate our method on a Gaussian data stream and its periodic extension, which is frequently observed in real-life data, as well as the conventional disjoint task-split. Our method outperforms prior arts by large margins in various setups, using four benchmark datasets in continual learning literature CIFAR-10, CIFAR-100, Tiny Image Net and Image Net. |
| Researcher Affiliation | Collaboration | 1Yonsei University 2GIST 3NAVER AI Lab 4NAVER Cloud 5Seoul National University |
| Pseudocode | Yes | Algorithm 1 provides detailed pseudocode for SDP. |
| Open Source Code | Yes | Code is available at https://github.com/yonseivnl/sdp. |
| Open Datasets | Yes | For empirical validations, following (Koh et al., 2021; Guo et al., 2022), we use four benchmark datasets; CIFAR-10, CIFAR-100, Tiny Image Net and Image Net. |
| Dataset Splits | No | The paper mentions 'confidence on currently learned model at the current time step before using them for training, as a proxy of validation accuracy' in Section 5.2, but it does not provide specific details on a separate validation dataset split (e.g., percentages, sample counts, or predefined splits) for reproducibility. |
| Hardware Specification | Yes | We compare the computational complexity by reporting the training time on a single NVIDIA Ge Force RTX 2080Ti. |
| Software Dependencies | No | The paper mentions using 'Res Net-18' and 'Adam optimizer' but does not specify version numbers for any software dependencies (e.g., Python, PyTorch, TensorFlow, CUDA versions). |
| Experiment Setup | Yes | For CIFAR-10, CIFAR-100, Tiny Image Net and Image Net, we use batch size of 16, 16, 32, 256, number of updates per sample of 1, 3, 3, 0.25, memory size of 500, 2000, 4000, 20000, respectively. We use Adam optimizer with LR of 0.0003 for all datasets and setup. |