Convolutional and Residual Networks Provably Contain Lottery Tickets
Authors: Rebekka Burkholz
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4. Experiments, We verify in experiments that our theory derives realistic conditions. The first target network type is a small scale example with 3 layers and is pruned and trained on MNIST (Deng, 2012). The second target that we consider has a more realistic structure that is much deeper and includes residual blocks. Similar to before, we prune and train Res Net-22 on CIFAR10 (Krizhevsky, 2009). |
| Researcher Affiliation | Academia | Rebekka Burkholz 1, 1CISPA Helmholtz Center for Information Security, Saarbr ucken, Germany. |
| Pseudocode | Yes | Pseudocode for the algorithm that constructs a LT to approximate a given target network is provided in Appendix F. Algorithm 1 2L approximation, Algorithm 2 L + 1 approximation |
| Open Source Code | Yes | The Github repository Relational ML/LT-existence contains a corresponding Pytorch implementation and all code for the experiments. |
| Open Datasets | Yes | The first target network type is a small scale example with 3 layers and is pruned and trained on MNIST (Deng, 2012). we prune and train Res Net-22 on CIFAR10 (Krizhevsky, 2009). |
| Dataset Splits | No | No specific dataset split information (exact percentages, sample counts, citations to predefined splits, or detailed splitting methodology) needed to reproduce the data partitioning was found for training/validation/test splits. |
| Hardware Specification | Yes | on a machine with Intel(R) Core(TM) i9-10850K CPU @ 3.60GHz processor and GPU NVIDIA Ge Force RTX 3080 Ti. |
| Software Dependencies | No | The Github repository Relational ML/LT-existence contains a corresponding Pytorch implementation and all code for the experiments. (No specific versions for Pytorch or other dependencies mentioned) |
| Experiment Setup | Yes | which we apply with exponential annealing of the target sparsity with 12 steps on a machine with Intel(R) Core(TM) i9-10850K CPU @ 3.60GHz processor and GPU NVIDIA Ge Force RTX 3080 Ti. Between pruning steps, we train the pruned network for 50 epochs. |