Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Variance Reduction With Sparse Gradients
Authors: Melih Elibol, Lihua Lei, Michael I. Jordan
ICLR 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, our algorithm consistently outperforms Spider Boost using various models on various tasks including image classification, natural language processing, and sparse matrix factorization. |
| Researcher Affiliation | Academia | Melih Elibol, Michael I. Jordan University of California, Berkeley EMAIL Lihua Lei Stanford University EMAIL |
| Pseudocode | Yes | Algorithm 1: Spider Boost with Sparse Gradients. |
| Open Source Code | No | The paper does not contain an explicit statement about releasing open-source code or provide a link to a code repository. |
| Open Datasets | Yes | For datasets, we use CIFAR-10 (Krizhevsky et al.), SVHN (Netzer et al., 2011), and MNIST (Le Cun & Cortes, 2010). |
| Dataset Splits | No | The paper does not explicitly provide details about training/validation/test dataset splits, such as specific percentages or sample counts. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used to run its experiments, such as specific GPU or CPU models. |
| Software Dependencies | No | The paper mentions software like Tensorflow and Pytorch but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | For all experiments, unless otherwise specified, we run Spider Boost and Sparse Spider Boost with a learning rate η = 0.1, large-batch size B = 1000, small-batch size b = 100, inner loop length of m = 10, memory decay factor of α = 0.5, and k1 and k2 both set to 5% of the total number of model parameters. |