Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?
Authors: Mansheej Paul, Feng Chen, Brett W. Larsen, Jonathan Frankle, Surya Ganguli, Gintare Karolina Dziugaite
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We do so through extensive empirical investigations on a range of benchmark datasets (CIFAR-10, CIFAR-100, and Image Net) and modern network architectures (Res Net-20, Res Net-18, and Res Net-50). Our contributions are as follows: |
| Researcher Affiliation | Collaboration | 1Stanford, 2Meta AI, 3Flatiron Institute, 4Mosaic ML, 5Harvard, 6Google Research, Brain Team, 7Mila; Mc Gill |
| Pseudocode | Yes | Algorithm 1: Iterative Magnitude Pruning-Weight Rewinding (IMP-WR) |
| Open Source Code | No | The paper does not state that its code is open source or provide a link to the code for the described methodology. |
| Open Datasets | Yes | We do so through extensive empirical investigations on a range of benchmark datasets (CIFAR-10, CIFAR-100, and Image Net) and modern network architectures (Res Net-20, Res Net-18, and Res Net-50). |
| Dataset Splits | No | The paper frequently refers to 'test error' but does not explicitly mention 'validation' splits or provide details for training/test/validation dataset splits. |
| Hardware Specification | No | The paper mentions 'Google Cloud research credits' and 'Meta AI' as funding/performance locations, but does not provide specific hardware details such as GPU/CPU models or specific cloud instance types. |
| Software Dependencies | No | The paper mentions 'Py Hessian package (Yao et al., 2020)' but does not provide a specific version number. Other software like 'SGD' are general algorithms without version details. |
| Experiment Setup | Yes | CIFAR-10 Res Net-20. We train with SGD and a batchsize of 128 for 62400 steps. We use lr = 0.1, momentum = 0.9, weight decay = 0.0001. The learning rate is decayed by a factor or 10 at 31200 and 46800 steps. |