Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processes
Authors: Liam Hodgkinson, Chris Van Der Heide, Fred Roosta, Michael W. Mahoney
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We verify empirically that our results hold for real data, beyond our considered assumptions, and we explore consequences involving synthetic covariates. Our theory is supported by experiments performed on real large datasets. |
| Researcher Affiliation | Academia | 1School of Mathematics and Statistics, University of Melbourne, Australia 2Department of Electrical and Electronic Engineering, University of Melbourne, Australia 3School of Mathematics and Physics, University of Queensland, Australia 4ARC Training Centre for Information Resilience, University of Queensland, Australia 5International Computer Science Institute 6Lawrence Berkeley National Laboratory 7Department of Statistics, University of California at Berkeley. |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | No | The paper does not provide an explicit statement or link for the open-sourcing of the code for the described methodology. |
| Open Datasets | Yes | To demonstrate our procedure for working with real data, we first consider the CT Slices dataset obtained from the UCI Machine Learning Repository (Graf et al., 2011)... We conducted parallel experiments on two larger benchmark datasets that are ubiquitous in the literature MNIST (Le Cun et al., 1998) and CIFAR10 (Krizhevsky & Hinton, 2009). |
| Dataset Splits | No | The paper mentions using subsets of datasets (e.g., 'n = 175 images were randomly sampled for use as the dataset' for MNIST), but does not specify train/validation/test splits or a cross-validation setup for its experiments. |
| Hardware Specification | No | The paper does not provide any specific hardware specifications (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies used in the experiments. |
| Experiment Setup | Yes | In each figure shown throughout this work, a performance metric has been calculated for varying dataset size n, input dimension d, and hyperparameters γ, λ. For experiments involving synthetic data, X Rn d has iid rows drawn from N(0, Σ), and Y = (Yi)n i=1 is comprised of iid samples from N(0, σ2) (where Σ = I and σ = 1 unless specified otherwise). For PPL2 and PPNLL, the expectation is computed over iid scalar test points x, y N(0, 1). Runs are averaged over a number of iterations, and 95% confidence intervals (under the central limit theorem) are highlighted. In Table 2 we present the parameters used for each figure. |