Function-Space Distributions over Kernels
Authors: Gregory Benton, Wesley J. Maddox, Jayson Salkey, Julio Albinati, Andrew Gordon Wilson
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the practicality of FKL over a wide range of experiments: (1) recovering known kernels from data (Section 5.1); (2) extrapolation (Section 5.2); (3) multi-dimensional inputs and irregularly spaced data (section 5.3); (4) multi-task precipitation data (Section 5.4); and (5) multidimensional pattern extrapolation (Section 5.5). We compare to the standard RBF and Matérn kernels, as well as spectral mixture kernels [34], and the Bayesian nonparametric spectral estimation (BNSE) of Tobar [30]. |
| Researcher Affiliation | Collaboration | Gregory W. Benton 1 Wesley J. Maddox 2 Jayson P. Salkey 1 Júlio Albinati 3 Andrew Gordon Wilson1,2 1Courant Institute of Mathematical Sciences, New York University 2Center for Data Science, New York University 3Microsoft |
| Pseudocode | Yes | A full description of the method is in Algorithm 1 in Appendix 2. Following the procedure outlined in Section 4 and detailed in Algorithm 2 in the Appendix... |
| Open Source Code | Yes | Code is available at https://github.com/wjmaddox/spectralgp . |
| Open Datasets | Yes | Airline Passenger Data We next consider the airline passenger dataset [14] consisting of 96 monthly observations of numbers of airline passengers from 1949 to 1961... We use the multi-task version of FKL in Section 3.4 to model precipitation data sourced from the United States Historical Climatology Network [19]. We use the product kernel described in Section 5.3 with both separate and shared latent GPs for regression tasks on UCI datasets. |
| Dataset Splits | Yes | We standardize the data to zero mean and unit variance and randomly split the training and test sets, corresponding to 90% and 10% of the full data, respectively. We conduct experiments over 10 random splits and show the average RMSE and standard deviation. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions 'Py Torch' and 'GPy Torch' as software used but does not provide specific version numbers for these libraries or any other software dependencies. |
| Experiment Setup | Yes | For FKL experiments, we use g(ω) with a negative quadratic mean function (to induce an RBF-like prior mean in the distribution over kernels), and a Matérn kernel with ν = 3/2 (to capture the typical sharpness of spectral densities). We use the heuristic for frequencies in the trapezoid rule described in Section 3.1. Using J = 10 samples from the posterior over kernels... This objective can be optimized using any procedure; we use the AMSGRAD variant of Adam as implemented in Py Torch [26]. |