Deep Gaussian Processes with Importance-Weighted Variational Inference
Authors: Hugh Salimbeni, Vincent Dutordoir, James Hensman, Marc Deisenroth
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our results demonstrate that the importance-weighted objective works well in practice and consistently outperforms classical variational inference, especially for deeper models. We investigate a large number of datasets and demonstrate that highly non Gaussian marginals occur in practice, and that they are not well modelled by the noise-free approach of Salimbeni and Deisenroth (2017). We also show that our importanceweighted scheme is always an improvement over variational inference, especially for the deeper models. We use 41 publicly available datasets with 1D targets. The datasets range in size from 23 points to 2, 049, 280. In each case we reserve 10% of the data for evaluating a test loglikelihood, repeating the experiment five times with different splits. We use five samples for the importance-weighted models, 128 inducing points, and five GP outputs for the inner layers. Hyperparameters and initializations are the same for all models and datasets and are fully detailed in the supplementary material. Results for test log-likelihood are reported in Table 1 for the GP models. |
| Researcher Affiliation | Collaboration | Hugh Salimbeni 1 2 Vincent Dutordoir 2 James Hensman 2 Marc Peter Deisenroth 1 2 1Imperial College London 2PROWLER.io. Correspondence to: Hugh Salimbeni <hrs13@ic.ac.uk>. |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | Full code to reproduce our results is available online 3. 3https://github.com/hughsalimbeni/DGPs_ with_IWVI |
| Open Datasets | Yes | We use 41 publicly available datasets2 with 1D targets. 2The full datasets with the splits and pre-processing can be found at github.com/hughsalimbeni/bayesian_ benchmarks. |
| Dataset Splits | Yes | In each case we reserve 10% of the data for evaluating a test loglikelihood, repeating the experiment five times with different splits. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used for the experiments. |
| Software Dependencies | No | The paper mentions 'Tensor Flow' in the acknowledgements (GPflow: a Gaussian Process Library using Tensor Flow), but it does not specify any software names with version numbers for reproducibility. |
| Experiment Setup | Yes | Hyperparameters and initializations are the same for all models and datasets and are fully detailed in the supplementary material. |