High-Dimensional Gaussian Process Inference with Derivatives
Authors: Filip de Roos, Alexandra Gessner, Philipp Hennig
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5. Experiments In the preceding section we outlined three applications where nonparametric models could benefit from efficient gradient inference in high dimensions. These ideas have been explored in previous work with the focus of improving traditional baselines, but always with various tricks to circumvent the expensive gradient inference. Since the purpose of this paper is to enable gradient inference and not develop new competing algorithms, the presented experiments are meant as a proof-of-concept to assess the feasibility of highdimensional gradient inference for these algorithms. |
| Researcher Affiliation | Academia | 1Department of Computer Science, University of T uebingen, T ubingen, Germany 2Max Planck Institute for Intelligent Systems, T ubingen, Germany. |
| Pseudocode | Yes | Algorithm 1 GP-[H/X] Optimization |
| Open Source Code | Yes | 1Code repository: https://github.com/fidero/gp-derivative |
| Open Datasets | No | The paper uses mathematical functions like 'quadratic function' (Eq. 14) and 'Rosenbrock function' (Eq. 17) which are defined analytically, not as publicly accessible datasets. For HMC, it uses a 'synthetic 100 dimensional target density'. No concrete access information (link, DOI, repository, or standard dataset citation) for a publicly available or open dataset is provided. |
| Dataset Splits | No | The paper does not provide specific details on training, validation, or test dataset splits (e.g., percentages, sample counts, or citations to predefined splits). It mentions 'retained all the observations' or 'last 2 observations for inference' for certain applications, but these do not constitute formal dataset split information for reproducibility. |
| Hardware Specification | Yes | The solver ran for 520 iterations until a relative tolerance of 10^-6 was reached, which took 4.9 seconds on a 2.2GHz 8-core processor. |
| Software Dependencies | No | The paper mentions 'scipy's BFGS implementation' but does not provide specific version numbers for scipy or any other software dependencies crucial for replication. |
| Experiment Setup | Yes | The nonparametric models use an isotropic RBF kernel with the last 2 observations for inference. All algorithms share the same line search routine. For this experiment a lengthscale of ℓ2 = 10 D was used with the isotropic RBF kernel, i.e., the inverse lengthscale matrix Λ = 10^-3 I. For training of GPG-HMC, we assign a budget N = D. |