GAUCHE: A Library for Gaussian Processes in Chemistry
Authors: Ryan-Rhys Griffiths, Leo Klarner, Henry Moss, Aditya Ravuri, Sang Truong, Yuanqi Du, Samuel Stanton, Gary Tom, Bojana Rankovic, Arian Jamasb, Aryan Deshwal, Julius Schwartz, Austin Tripp, Gregory Kell, Simon Frieder, Anthony Bourached, Alex Chan, Jacob Moss, Chengzhi Guo, Johannes Peter Dürholt, Saudamini Chaurasia, Ji Won Park, Felix Strieth-Kalthoff, Alpha Lee, Bingqing Cheng, Alan Aspuru-Guzik, Philippe Schwaller, Jian Tang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate GAUCHE on a range of regression, uncertainty quantification (UQ) and Bayesian optimisation (BO) tasks. The principle goal in conducting regression and UQ benchmarks is to gauge whether performance on these tasks may be used as a proxy for BO performance. We make use of the following datasets with experimentally determined labels: Photoswitch, ESOL, Free Solv, Lipophilicity, Buchwald-Hartwig reactions, Suzuki-Miyaura reactions. For the regression and uncertainty quantification experiments, all datasets were randomly split into training and test sets with a ratio of 80/20. To quantify model performance, the predictive accuracy and the calibration of the predictive uncertainty estimates of the fitted models were evaluated on the held-out test set and summarised as the root-mean-square error (RMSE) and the negative log predictive density (NLPD), respectively. The mean and standard error of these metrics over 20 different random splits are reported in Table 1. |
| Researcher Affiliation | Collaboration | 1Meta 2University of Oxford 3University of Cambridge 4Stanford University 5Genentech 6University of Toronto 7Vector Institute 8EPFL 9NCCR Catalysis 10Cornell University 11Washington State University 12King s College London 13University College London 14Evonik Industries AG 15Syracuse University 16IST Austria 17CIFAR AI Research Chair 18MILA Quebec AI Institute 19HEC Montreal |
| Pseudocode | Yes | We provide an example of the class definition for the Tanimoto kernel in GAUCHE below... class Tanimoto GP(Exact GP): ... and an example definition of a black box kernel ... class WLKernel(gauche.Kernel): ... |
| Open Source Code | Yes | The codebase is made available at https://github.com/leojklarner/gauche. |
| Open Datasets | Yes | We make use of the following datasets with experimentally determined labels: Photoswitch [3], ESOL [60], Free Solv [41], Lipophilicity [61, 62], Buchwald-Hartwig reactions [41], Suzuki-Miyaura reactions [63]. |
| Dataset Splits | Yes | For the regression and uncertainty quantification experiments, all datasets were randomly split into training and test sets with a ratio of 80/20. (Note that validation sets are not required for the GP models, since hyperparameters are chosen using the marginal likelihood objective on the train set). |
| Hardware Specification | No | The paper mentions 'GPU-accelerated' kernels but does not provide specific details about the hardware (e.g., GPU model, CPU, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions libraries such as GPy Torch, Bo Torch, and Gra Kel, and the L-BFGS-B optimiser, but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | All GP models were trained using the L-BFGS-B optimiser [64] and, if not stated otherwise, the default settings in the GPy Torch and Bo Torch libraries apply. We choose a radius of 3 for all experiments in the main text. For the sub-string count featurisation ..., we set n = 5 in our experiments. BO is run for 20 iterations of sequential candidate selection (EI acquisition) where candidates are drawn from 95% of the dataset. The models are initialised with 5% of the dataset. |