Learning to Boost Training by Periodic Nowcasting Near Future Weights
Authors: Jinhyeok Jang, Woo-Han Yun, Won Hwa Kim, Youngwoo Yoon, Jaehong Kim, Jaeyeon Lee, Byungok Han
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that WNN can significantly save actual time cost for training with an additional marginal time to train WNN. We validate the generalization capability of WNN under various tasks, and demonstrate that it works well even for unseen tasks. |
| Researcher Affiliation | Academia | 1ETRI. 2KAIST. 3POSTECH. |
| Pseudocode | Yes | Algorithm 1 Train Simulation with Varying Θtarget |
| Open Source Code | Yes | The code and pre-trained model are available at https://github.com/jjh6297/WNN. |
| Open Datasets | Yes | As datasets, we adopted CIFAR10 (Krizhevsky et al., 2009) and MNIST |
| Dataset Splits | Yes | We trained the Vanilla CNN on CIFAR10 which consists of 50,000 images of 32 × 32 with 10 classes for training and 10,000 images for validation. |
| Hardware Specification | Yes | NVIDIA TITAN Xp GPU was used to estimate time cost. |
| Software Dependencies | No | The paper mentions software like “Py GYM library” and “Adam optimizer” but does not provide specific version numbers for any software, frameworks, or libraries used in the experiments. |
| Experiment Setup | Yes | Both Res Net and Vanilla CNN were trained by Adam optimizer with a fixed learning rate (LR) of 1e-3, 0.9 and 0.999 as two βs without LR scheduling, 1,024 batch size, simple data augmentations (random flip, rotation, translation) for totally 200 epochs. |