Lookahead: A Far-sighted Alternative of Magnitude-based Pruning
Authors: Sejun Park*, Jaeho Lee*, Sangwoo Mo, Jinwoo Shin
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experimental results demonstrate that the proposed method consistently outperforms magnitude-based pruning on various networks, including VGG and Res Net, particularly in the high-sparsity regime. |
| Researcher Affiliation | Academia | KAIST EE KAIST AI {sejun.park,jaeho-lee,swmo,jinwoos}@kaist.ac.kr |
| Pseudocode | Yes | Algorithm 1 Lookahead Pruning (LAP) |
| Open Source Code | Yes | See https://github.com/alinlab/lookahead_pruning for codes. |
| Open Datasets | Yes | FCN is trained on MNIST dataset (Lecun et al., 1998), Conv-6, VGG, and Res Net are trained on CIFAR-10 dataset (Krizhevsky & Hinton, 2009), and VGG, Res Net, and WRN are trained on Tiny-Image Net. |
| Dataset Splits | No | The paper mentions 'test accuracy' and 'retraining phase' but does not explicitly provide information on training/test/validation splits with percentages, sample counts, or citations to predefined validation splits. |
| Hardware Specification | Yes | The computations have taken place on 40 CPUs of Intel Xeon E5-2630v4 @ 2.20GHz. ...additionally used a single NVidia Ge Force GTX1080 |
| Software Dependencies | No | The paper mentions 'Py Torch' and 'Back PACK' but does not provide specific version numbers for these software dependencies. It mentions 'Adam optimizer' but this is an algorithm, not a software dependency with a version. |
| Experiment Setup | Yes | We use Adam optimizer (Kingma & Ba, 2015) with batch size 60. We use a learning rate of 1.2 10 3 for FCN and 3 10 4 for all other models. For FCN, we use [50k, 50k] for the initial training phase and retraining phase. For Conv-6, we use [30k, 20k] steps. |