Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Approximation Based Variance Reduction for Reparameterization Gradients
Authors: Tomas Geffner, Justin Domke
NeurIPS 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically show that this control variate leads to large improvements in gradient variance and optimization convergence for inference with non-factorized variational distributions. |
| Researcher Affiliation | Academia | Tomas Geffner College of Information and Computer Science University of Massachusetts, Amherst EMAIL Justin Domke College of Information and Computer Science University of Massachusetts, Amherst EMAIL |
| Pseudocode | Yes | Algorithm 1 SGVI with the proposed control variate. |
| Open Source Code | No | The paper does not provide an explicit statement or link for open-source code for the methodology. |
| Open Datasets | Yes | We use three different models: Logistic regression with the a1a dataset, hierarchical regression with the frisk dataset [7], and a Bayesian neural network with the red wine dataset. The latter two are the ones used by Miller et al. [17]. |
| Dataset Splits | No | The paper does not explicitly provide training/validation/test dataset splits needed to reproduce the experiment. |
| Hardware Specification | Yes | we use Py Torch 1.1.0 on an Intel i5 2.3GHz |
| Software Dependencies | Yes | we use Py Torch 1.1.0 on an Intel i5 2.3GHz |
| Experiment Setup | Yes | We use Adam [13] to optimize the parameters w of the variational distribution qw (with step sizes between 10 5 and 10 2). We use Adam with a step size of 0.01 to optimize the parameters v of the control variate, by minimizing the proxy to the variance from Eq. 12. We parameterize Bv as a diagonal plus rank-rv. We set rv = 10 when diagonal or diagonal plus low rank variational distributions are used, and rv = 20 when a full-rank variational distribution is used. We use M = 10 and M = 50 samples from qw to estimate gradients. |