Differentiable Annealed Importance Sampling and the Perils of Gradient Noise
Authors: Guodong Zhang, Kyle Hsu, Jianing Li, Chelsea Finn, Roger B. Grosse
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our theoretical analysis with simulations. We also demonstrate empirically that DAIS can be applied to variational autoencoders (VAEs) [Kingma and Welling, 2013, Rezende et al., 2014] for a tighter evidence lower bound, which in turn leads to improved performance compared to the vanilla VAE. |
| Researcher Affiliation | Academia | 1University of Toronto, 2Vector Institute, 3Stanford University |
| Pseudocode | Yes | Algorithm 1 Differentiable AIS (DAIS) |
| Open Source Code | No | The paper does not provide an explicit statement or link for open-source code for the described methodology. |
| Open Datasets | Yes | We use the dynamically binarized MNIST [Le Cun et al., 1998] dataset. |
| Dataset Splits | No | The paper does not provide specific details about dataset splits (e.g., percentages or sample counts for training, validation, and test sets), nor does it explicitly mention a validation set with its partitioning. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used for running its experiments, such as specific GPU or CPU models. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies used in the experiments. |
| Experiment Setup | Yes | We use the same architecture as in IWAE paper. The prior p(z) is a 50-dimensional standard Gaussian distribution. The conditional distributions p(xi|z) are independent Bernoulli, with the decoder parameterized by two hidden layers, each with 200 tanh units. The variational posterior q(z|x) is also a 50-dimensional Gaussian with diagonal covariance, whose mean and variance are both parameterized by two hidden layers with 200 tanh units (see other details in Appendix C.1). By default, we use partial momentum refreshment with γ = 0.9 and equally spaced annealing parameters βk = k/K. |