Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference
Authors: Jonathan Wenger, Kaiwen Wu, Philipp Hennig, Jacob Gardner, Geoff Pleiss, John P. Cunningham
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that model selection for computation-aware GPs trained on 1.8 million data points can be done within a few hours on a single GPU. |
| Researcher Affiliation | Academia | Jonathan Wenger1 Kaiwen Wu2 Philipp Hennig3 Jacob R. Gardner2 Geoff Pleiss4 John P. Cunningham1 1 Columbia University 2 University of Pennsylvania 3 University of T ubingen, T ubingen AI Center 4 University of British Columbia, Vector Institute |
| Pseudocode | Yes | We give algorithms both for iteratively constructed dense actions (Algorithm S1), as used in Ca GPCG, and for sparse batch actions (Algorithm S2), as used for Ca GP-Opt, in the supplementary material. |
| Open Source Code | Yes | We provide an open-source implementation of Ca GP-Opt as part of GPy Torch. To install the package via pip, execute the following in the command line: pip install git+https://github.com/cornellius-gp/linear_operator.git@sparsity pip install git+https://github.com/cornellius-gp/gpytorch.git@computation-aware-gps-v2 pip install pykeops |
| Open Datasets | Yes | We benchmark the generalization of computation-aware GPs with two different action choices, Ca GP-Opt (ours) and Ca GP-CG [19], using our proposed training objective in Equation (10) on a range of UCI datasets for regression [53]. |
| Dataset Splits | Yes | All datasets were randomly partitioned into train and test sets using a (0.9, 0.1) split for five random seeds. |
| Hardware Specification | Yes | All experiments were run on an NVIDIA Tesla V100-PCIE-32GB GPU, except for Power , where we used an NVIDIA A100 80GB PCIe GPU to have sufficient memory for Ca GP-Opt with i = 512. |
| Software Dependencies | No | We used the existing implementations of SGPR, SVGP and CGGP in GPy Torch [7] and also implemented Ca GP in this framework (see Section S4.2 for our open-source implementation). ... pip install pykeops |
| Experiment Setup | Yes | We used a zero-mean GP prior and a Mat ern(3/2) kernel with an outputscale o2 and one lengthscale per input dimension l2 j, as well as a scalar observation noise for the likelihood σ2, s.t. θ = (o, l1, . . . , ld, σ) Rd+2. ... For SGPR and SVGP we used m = 1024 inducing points and for CGGP, Ca GP-CG and Ca GP-Opt we chose i = 512. We optimized the hyperparameters θ either with Adam [54] for a maximum of 1000 epochs in float32 or with LBFGS [55] for 100 epochs in float64, depending on the method and problem scale. |