Uncertainty-based Continual Learning with Adaptive Regularization
Authors: Hongjoon Ahn, Sungmin Cha, Donggyu Lee, Taesup Moon
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive experiments, we show UCL convincingly outperforms most of recent state-of-the-art baselines not only on popular supervised learning benchmarks, but also on challenging lifelong reinforcement learning tasks. |
| Researcher Affiliation | Academia | Hongjoon Ahn1 , Sungmin Cha2 , Donggyu Lee2 and Taesup Moon1,2 1 Department of Artificial Intelligence, 2Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea 16419 {hong0805, csm9493, ldk308, tsmoon}@skku.edu |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The source code of our algorithm is available at https://github.com/csm9493/UCL. |
| Open Datasets | Yes | For the experiments with MNIST datasets, we used fully-connected neural networks (FNN), and with CIFAR-10/100 and Omniglot datasets, we used convolutional neural networks (CNN). Roboschool [24] consists of 12 tasks... |
| Dataset Splits | No | The paper mentions training and testing, and refers to 'Supplementary Materials' for details on architectures and hyperparameters, but does not explicitly provide specific train/validation/test dataset split percentages or sample counts in the main body. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependency details, such as library names with version numbers, required to replicate the experiments. |
| Experiment Setup | Yes | The initial standard deviations for UCL, {σ(l) init}L l=1, were set to be 0.06 for FNNs and adaptively set like the He initialization [7] for deeper CNNs, of which details are given in the Supplementary Material. The hyperparameter selections among the baselines are done fairly, and we list the selected hyperparameters in the Supplementary Materials. Note we show two versions of UCL, with different β hyperparameter values. UCL ( (l) init = 5x10 3, = 5x10 5) UCL ( (l) init = 5x10 3, = 5x10 6) |