Bayesian leave-one-out cross-validation for large data
Authors: Måns Magnusson, Michael Andersen, Johan Jonasson, Aki Vehtari
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To study the characteristic of the proposed approach we study multiple models and datasets. We use simulated datasets used to fit a Bayesian linear regression model with D variables and N observations. The data is generated such that so we get either a correlated (c) or an independent (i) posterior for the regression parameters by construction. This will enable us to study the effect of the mean-field assumptions in variational posterior approximations. In addition, we use data from the radon example of Lin et al. (1999) to show performance on a larger dataset with multiple models. |
| Researcher Affiliation | Academia | 1Department of Computer Science, Aalto University, Finland 2Department of Applied Mathematics and Computer Science, Technical University of Denmark, Denmark 3Department of Mathematical Sciences, Chalmers University of Technology and University of Gothenburg, Sweden. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. The method summary describes steps in numbered list format, but it is not pseudocode. |
| Open Source Code | Yes | The methods have been implemented using the loo R package (Vehtari et al., 2018) framework for Stan and are available as supplementary material. |
| Open Datasets | Yes | We use simulated datasets used to fit a Bayesian linear regression model with D variables and N observations. ... In addition, we use data from the radon example of Lin et al. (1999) |
| Dataset Splits | Yes | Leave-one-out cross-validation (LOO-CV) is one approach to estimate the elpd for a given model, and is the method of focus in this paper (Bernardo & Smith, 1994; Vehtari & Ojanen, 2012; Vehtari et al., 2017). ... As a comparison we also compute elpd10fcv, computing an estimate of elpd using a 10-fold cross-validation scheme, without bias correction. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. It mentions 'computer resources within the Aalto University School of Science Science-IT project' but no specifics. |
| Software Dependencies | Yes | All posterior computations use Stan 2.18 (Carpenter et al., 2017; Stan Development Team, 2018) and all models used can be found in the supplementary material. The methods have been implemented using the loo R package (Vehtari et al., 2018) framework for Stan and are available as supplementary material. We use mean-field and full-rank Automatic Differentiation Variational Inference (ADVI, Kucukelbir et al., 2017) and Laplace approximations as implemented in Stan. |
| Experiment Setup | Yes | We used 100 000 iterations for ADVI and 1000 warmup iterations and 2000 samples from 4 chains for the MCMC. |