Bayesian Optimization of Function Networks with Partial Evaluations
Authors: Poompol Buathong, Jiayue Wan, Raul Astudillo, Sam Daulton, Maximilian Balandat, Peter I. Frazier
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the benefits of exploiting partial evaluations through several numerical experiments, including both synthetic and real-world applications with a variety of network structures. We evaluate p-KGFN against several benchmarks, including three algorithms that do not leverage the objective s function network structure: a random sampling baseline (Random), standard versions of expected improvement (EI) and knowledge gradient (KG), and three algorithms that do leverage network structure but require evaluation of the full network: EIFN (Astudillo & Frazier, 2021a), a slight modification of EIFN that uses the knowledge gradient instead of EI (KGFN), and Thompson sampling for function networks (TSFN). |
| Researcher Affiliation | Collaboration | 1Center for Applied Mathematics, Cornell University 2School of Operations Research and Information Engineering, Cornell University 3Department of Computing and Mathematical Sciences, Caltech 4Meta. |
| Pseudocode | Yes | A. Pseudo-Code for the p-KGFN Algorithm We present the pseudo-code for implementing Bayesian optimization with the p-KGFN acquisition function, supplementing the descriptions in Section 5. Algorithm 1 outlines the BO loop employing the p-KGFN algorithm. Algorithm 2 describes the computation of the MC estimate of the acquisition value. Algorithm 3 describes how we estimate the posterior mean of the final function node via MC simulation, which is necessary for Algroithm 2. |
| Open Source Code | Yes | All algorithms were implemented in Bo Torch (Balandat et al., 2020). The code to reproduce our experiments is available at https: //github.com/frazier-lab/partial_kgfn. |
| Open Datasets | Yes | Molecular Design (Free Solv) We consider the Free Solv dataset (Mobley & Guthrie, 2014), which consists of calculated and experimental hydration-free energies of 642 small molecules. Pharmaceutical Product Development (Pharma) ... We employ surrogate models proposed in Sano et al. (2020) for these two target properties as functions of four input variables in the production process. |
| Dataset Splits | No | The paper describes initial observations for fitting GP models and subsequent sequential evaluations, but does not specify a conventional training/validation/test dataset split for reproducing experiments. |
| Hardware Specification | Yes | In this section, we report wall clock time on 8-core CPUs used to optimize each acquisition function on Ackley experiment. |
| Software Dependencies | No | All algorithms were implemented in the open source Bo Torch package (Balandat et al., 2020). While Bo Torch is named, a specific version number is not provided, nor are other key software dependencies with their versions (e.g., Python, PyTorch, NumPy versions). |
| Experiment Setup | Yes | In our experiments, all methods utilize independent GPs with zero mean functions and the Mate rn 5/2 kernel (Genton, 2001), with automatic relevance determination (ARD). The lengthscales of the GPs are assumed to have Gamma priors: for the Ackley, Free Solv and Pharm problems, Gamma(3, 6); for the Manu-GP problem, Gamma(5, 2). The outputscale parameters are assumed to have Gamma(2, 0.15) priors in all problems. The lengthscales and outputscales are then estimated via maximum a posteriori (MAP) estimation. In p-KGFN, we estimate the posterior mean of a function network s final node value with J = 64 quasi-MC samples using Sobol sequences (Balandat et al., 2020). For EIFN, we follow the implementation in Astudillo & Frazier (2021a), using J = 128. To compute MC estimates for the p-KGFN acquisition value, we use I = 8 fantasy models. ... We include in A the current maximizer of the final node posterior mean x n, NT = 10 points obtained from a Thompson sampling method described in the main text, NL = 10 local points with r = 0.1. |