Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Dual Parameterization of Sparse Variational Gaussian Processes
Authors: Vincent ADAM, Paul Chang, Mohammad Emtiyaz Khan, Arno Solin
NeurIPS 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Empirical Evaluation We conduct experiments to highlight the advantages of using the dual parameterization. Firstly, we study the effects of the improved objective for hyperparameter learning of t-SVGP versus q-SVGP. We study the objective being optimized for a single M-step, after an E-step ran until convergence. We then show a full sequence of EM iterations on small data sets. For large-scale data, where running steps to convergence is expensive, we use partial E and M-steps and mini-batching. Our improved bound and faster natural gradient computations show benefits in both settings. |
| Researcher Affiliation | Collaboration | Vincent Adam Aalto University / Secondmind.ai Espoo, Finland / Cambridge, UK EMAIL Paul E. Chang Aalto University Espoo, Finland EMAIL Mohammad Emtiyaz Khan RIKEN Center for AI Project Tokyo, Japan EMAIL Arno Solin Aalto University Espoo, Finland EMAIL |
| Pseudocode | Yes | The full algorithm is given in App. E. |
| Open Source Code | Yes | We provide a reference implementation of our method under the GPflow framework at https: //github.com/Aalto ML/t-SVGP. |
| Open Datasets | Yes | MNIST ([23], available under CC BY-SA 3.0), We use common small and mid-sized UCI data sets to test the performance of our method |
| Dataset Splits | Yes | We perform 5-fold cross validation with the results in Fig. 3 showing the mean of the folds for ELBO and NLPD. |
| Hardware Specification | Yes | We compare wall-clock time to compute 150 steps of the algorithm for both methods in terms of NLPD and ELBO taking single E and M-steps (Mac Book pro, 2 GHz CPU, 16 GB RAM). |
| Software Dependencies | Yes | We compare against the state-of-the-art implementation of SVGP in GPflow ([26], v2.2.1) |
| Experiment Setup | Yes | All experiments are performed with a batch size of nb = 200 and m = 100 inducing points and the optimization is ran until convergence using the Adam optimizer for the hyperparameters (M-step)., Table 1: NLPD on MNIST benchmarks for different learning rates and E and M steps. |