Debiasing Evidence Approximations: On Importance-weighted Autoencoders and Jackknife Variational Inference
Authors: Sebastian Nowozin
ICLR 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We now empirically validate our key claims regarding the JVI method: 1. JVI produces better estimates of the marginal likelihood by reducing bias, even for small K; and 2. Higher-order bias reduction is more effective than lower-order bias reduction; To this end we will use variational autoencoders trained on MNIST. |
| Researcher Affiliation | Industry | Sebastian Nowozin Machine Intelligence and Perception Microsoft Research, Cambridge, UK Sebastian.Nowozin@microsoft.com |
| Pseudocode | Yes | Algorithm 1 Computing ˆLJ,m K , the jackknife variational inference estimator |
| Open Source Code | Yes | 1The implementation is available at https://github.com/Microsoft/ jackknife-variational-inference |
| Open Datasets | Yes | Our setup is purposely identical to the setup of Tomczak & Welling (2016), where we use the dynamically binarized MNIST data set of Salakhutdinov & Murray (2008). |
| Dataset Splits | Yes | We train on the first 50,000 training images, using 10,000 images for validation. |
| Hardware Specification | Yes | All our models are implemented using Chainer (Tokui et al., 2015) and run on a NVidia Titan X. |
| Software Dependencies | No | The paper mentions 'Chainer (Tokui et al., 2015)' but does not specify a version number for this software dependency. |
| Experiment Setup | Yes | Hyperparameters are the batch size in {1024, 4096} and the SGD step size in {0.1, 0.05, 0.01, 0.005, 0.001}. |