REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
Authors: George Tucker, Andriy Mnih, Chris J. Maddison, John Lawson, Jascha Sohl-Dickstein
NeurIPS 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our experiments, we illustrate the potential problems inherent with biased gradient estimators on a toy problem. Then, we use REBAR to train generative sigmoid belief networks (SBNs) on the MNIST and Omniglot datasets and to train conditional generative models on MNIST. Across tasks, we show that REBAR has state-of-the-art variance reduction which translates to faster convergence and better final log-likelihoods. |
| Researcher Affiliation | Collaboration | 1Google Brain, 2Deep Mind, 3University of Oxford |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled as "Pseudocode" or "Algorithm". |
| Open Source Code | Yes | Source code for experiments: github.com/tensorflow/models/tree/master/research/rebar |
| Open Datasets | Yes | We used the statically binarized MNIST digits from Salakhutdinov & Murray (2008) and a fixed binarization of the Omniglot character dataset. |
| Dataset Splits | Yes | We used the standard splits into training, validation, and test sets. |
| Hardware Specification | No | The paper does not specify any hardware components (e.g., CPU, GPU models, or specific machine types) used for running the experiments. |
| Software Dependencies | No | The paper does not explicitly list any software dependencies with specific version numbers (e.g., Python 3.x, TensorFlow x.x). |
| Experiment Setup | No | The paper provides general model architecture details (e.g., linear vs. nonlinear layers) and notes that learning rates were optimized, but it does not provide specific hyperparameter values like learning rates, batch sizes, or number of epochs in the main text. |