Finding trainable sparse networks through Neural Tangent Transfer
Authors: Tianlin Liu, Friedemann Zenke
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4. Experiments We numerically evaluated the trainability of sparse networks found by the NTT algorithm for both MLPs and CNNs on standard datasets, including MNIST (Le Cun et al., 1998), Fashion MNIST (Xiao et al., 2017), CIFAR-10 (Krizhevsky & Hinton, 2009), and SVHN (Netzer et al., 2011). |
| Researcher Affiliation | Academia | 1Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland. 2University of Basel, Basel, Switzerland. |
| Pseudocode | No | The paper describes the 'Algorithmic implementation of NTT' in Section 3.3 using numbered steps, but this is presented as a procedural description rather than structured pseudocode or an algorithm block. |
| Open Source Code | No | The paper mentions using JAX and neural-tangents libraries developed by other teams, but it does not contain an explicit statement about releasing its own source code for the methodology described, nor does it provide a link. |
| Open Datasets | Yes | 4. Experiments We numerically evaluated the trainability of sparse networks found by the NTT algorithm for both MLPs and CNNs on standard datasets, including MNIST (Le Cun et al., 1998), Fashion MNIST (Xiao et al., 2017), CIFAR-10 (Krizhevsky & Hinton, 2009), and SVHN (Netzer et al., 2011). |
| Dataset Splits | No | The paper mentions training and testing datasets (e.g., 'Learning curves of NTT and baseline networks trained on MNIST', 'Test accuracy') but does not explicitly describe a separate validation set or its split. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | All experiments were performed in JAX (Bradbury et al., 2018) together with the neural-tangent library (Novak et al., 2020). While software names are provided, specific version numbers for JAX and neural-tangent are not mentioned. |
| Experiment Setup | Yes | For all experiments in this article, we used the procedure outlined above. The speciļ¬c experimental setup and all hyper-parameter choices are provided in the corresponding experiment sections and Appendix B. |