Proving the Lottery Ticket Hypothesis: Pruning is All You Need
Authors: Eran Malach, Gilad Yehudai, Shai Shalev-Schwartz, Ohad Shamir
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We prove an even stronger hypothesis (as was also conjectured in Ramanujan et al., 2019), showing that for every bounded distribution and every target network with bounded weights, a sufficiently over-parameterized neural network with random weights contains a subnetwork with roughly the same accuracy as the target network, without any further training. Our work aims to give theoretical evidence to these empirical results. We prove the latter conjecture, stated in (Ramanujan et al., 2019), in the case of deep and shallow neural networks. To the best of our knowledge, this is the first theoretical work aiming to explain the strong lottery ticket conjecture, as stated in (Ramanujan et al., 2019). |
| Researcher Affiliation | Academia | 1School of Computer Science, Hebrew University 2Weizmann Institute of Science. |
| Pseudocode | No | The paper provides theoretical proofs and discussions, but it does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement or link indicating the availability of open-source code for the described methodology. |
| Open Datasets | No | This is a theoretical paper and does not conduct experiments involving dataset training. |
| Dataset Splits | No | This is a theoretical paper and does not conduct experiments involving dataset validation. |
| Hardware Specification | No | This is a theoretical paper and does not report on experiments requiring hardware specifications. |
| Software Dependencies | No | This is a theoretical paper and does not report on experiments requiring software dependencies. |
| Experiment Setup | No | This is a theoretical paper and does not report on experiments requiring an experimental setup description. |