Conformal Bayesian Computation
Authors: Edwin Fong, Chris C Holmes
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the utility on a range of examples including extensions to partially exchangeable settings such as hierarchical models. |
| Researcher Affiliation | Academia | Edwin Fong University of Oxford The Alan Turing Institute edwin.fong@stats.ox.ac.uk Chris Holmes University of Oxford The Alan Turing Institute cholmes@stats.ox.ac.uk |
| Pseudocode | Yes | Algorithm 1: Full Conformal Prediction |
| Open Source Code | Yes | The code is available online1 (Footnote 1: https://github.com/edfong/conformal_bayes) |
| Open Datasets | Yes | We first demonstrate our method under a sparse linear regression model on the diabetes dataset (Efron et al., 2004)... We also analyze the Wisconsin breast cancer (Wolberg and Mangasarian, 1990)... and an application to the radon dataset of Gelman and Hill (2006) is given in Appendix D.4.2. |
| Dataset Splits | Yes | To check coverage, we repeatedly divide into a training and test dataset for 50 repeats, with 30% of the dataset in the test split. (Section 4.1) ... We again have 50 repeats with 70-30 train-test split, and set α = 0.2. (Section 4.2) ... For each of the 50 repeats, we draw nj = 10 training and test data points from each group (Section 4.3) |
| Hardware Specification | Yes | We run and time all examples on an Azure NC6 Virtual Machine, which has 6 Intel Xeon E5-2690 v3 v CPUs and a one-half Tesla K80 GPU card. |
| Software Dependencies | No | The paper mentions software like Py MC3, sklearn, and JAX but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | We evaluate the conformal prediction set on a grid of size ngrid = 100 between [ymin 2, ymax + 2]... We compute the central (1 α) credible interval from the Bayesian posterior predictive CDF estimated using Monte Carlo and the same grid as for CB. ... MCMC induced an average overhead of 21.9s for a = 1 and 26.8s for c = 0.02 for the Bayes and CB interval, where we simulate T = 8000 posterior samples. |