Towards Adaptive Residual Network Training: A Neural-ODE Perspective
Authors: Chengyu Dong, Liyuan Liu, Zichao Li, Jingbo Shang
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our experiments, it achieves comparable performance while reducing 50% of training time. |
| Researcher Affiliation | Academia | 1University of California, San Diego 2University of Illinois at Urbana-Champaign. |
| Pseudocode | Yes | Algorithm 1 presents a generic adaptive training setup. Algorithm 2 Our Lip Grow Algorithm |
| Open Source Code | Yes | 1https://github.com/shwinshaker/LipGrow |
| Open Datasets | Yes | We conduct experiments on three benchmark datasets, CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009) 5, and Tiny-Image Net6. 5www.cs.toronto.edu/~kriz/cifar.html 6www.kaggle.com/c/tiny-imagenet |
| Dataset Splits | Yes | We follow their standard train, validation, and test splits in our experiments. |
| Hardware Specification | Yes | evaluated on a single Nvidia Ge Force GTX 1080 Ti GPU. All models on the Tiny-Image Net dataset are trained for 90 epochs, and evaluated on a single Nvidia Quadro RTX 8000 GPU. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers. |
| Experiment Setup | Yes | All models on the CIFAR datasets are trained for 164 epochs... We use a batch size of 128 for training, and 100 for validation. Weight decay and momentum are set to be 2 10 4 and 0.9, respectively. |