Deep Probabilistic Programming
Authors: Dustin Tran, Matthew D. Hoffman, Rif A. Saurous, Eugene Brevdo, Kevin Murphy, David M. Blei
ICLR 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we illustrate two main benefits of Edward: flexibility and efficiency. For the former, we show how it is easy to compare different inference algorithms on the same model. For the latter, we show how it is easy to get significant speedups by exploiting computational graphs. |
| Researcher Affiliation | Collaboration | Dustin Tran Columbia University Matthew D. Hoffman Adobe Research Rif A. Saurous Google Research Eugene Brevdo Google Brain Kevin Murphy Google Research David M. Blei Columbia University |
| Pseudocode | Yes | Figure 1: Beta-Bernoulli program (left) alongside its computational graph (right). ... Figure 2: Variational auto-encoder for a data set of 28 28 pixel images: (left) graphical model, with dotted lines for the inference model; (right) probabilistic program, with 2-layer neural networks. ... Figure 3: Bayesian RNN: (left) graphical model; (right) probabilistic program. |
| Open Source Code | Yes | 1See Tran et al. (2016a) for details of the API. A companion webpage for this paper is available at http://edwardlib.org/iclr2017. It contains more complete examples with runnable code. |
| Open Datasets | Yes | We consider the VAE setup from Figure 2 and the binarized MNIST data set (Salakhutdinov & Murray, 2008). ... We apply logistic regression on the Covertype dataset (N = 581012, D = 54; responses were binarized) |
| Dataset Splits | No | While the paper mentions 'held-out log likelihoods' in Section 5.1, it does not specify the explicit percentages or sample counts for training, validation, or test splits. For example: 'After training we evaluate held-out log likelihoods, which are lower bounds on the true value.' |
| Hardware Specification | Yes | We benchmark runtimes for a fixed number of Hamiltonian Monte Carlo (HMC; Neal, 2011) iterations on modern hardware: a 12-core Intel i7-5930K CPU at 3.50GHz and an NVIDIA Titan X (Maxwell) GPU. |
| Software Dependencies | No | The paper mentions software like Edward, TensorFlow, Keras, Stan, and PyMC3 but does not provide specific version numbers for these software components. For example, it states 'Edward is integrated into Tensor Flow' and 'The program uses Keras (Chollet, 2015)'. |
| Experiment Setup | Yes | We use d = 50 latent variables per data point and optimize using ADAM. ... We ran 100 HMC iterations, with 10 leapfrog updates per iteration, a step size of 0.5/N, and single precision. |