Generative Adversarial Model-Based Optimization via Source Critic Regularization
Authors: Michael Yao, Yimeng Zeng, Hamsa Bastani, Jacob Gardner, James Gee, Osbert Bastani
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experimental Evaluation" and "We show that compared to prior methods, our proposed algorithm with Bayesian optimization empirically achieves the highest rank of 3.8 (second best is 5.5) on top-1 design evaluation, and highest rank of 3.0 (second best is 4.6) on top-128 design evaluation across a variety of tasks spanning multiple scientific domains. |
| Researcher Affiliation | Academia | Michael S. Yao Department of Bioengineering Perelman School of Medicine University of Pennsylvania Yimeng Zeng Department of Computer and Information Science University of Pennsylvania Hamsa Bastani The Wharton School University of Pennsylvania Jacob Gardner Department of Computer and Information Science University of Pennsylvania James C. Gee Department of Radiology University of Pennsylvania Osbert Bastani Department of Computer and Information Science University of Pennsylvania |
| Pseudocode | Yes | Algorithm 1 Adaptive Source Critic Regularization (SCR) Algorithm 2 Generative Adversarial Bayes Opt (GABO) |
| Open Source Code | Yes | Our code is available at https://github.com/michael-s-yao/gabo. All data and code associated with this paper is open access and includes sufficient instructions to faithfully reproduce the experimental results reported herein. We have made the data and code available in the Supplementary Material ZIP file. |
| Open Datasets | Yes | We use the publicly available Guacamol benchmarking dataset from Brown et al. (2019) to implement this task. Tasks (3) (7) are derived from Design-Bench, a publicly available set of MBO benchmarking tasks (Trabucco et al., 2022): Finally, (8) the Warfarin task uses the dataset of patients on warfarin medication from Consortium (2009) |
| Dataset Splits | Yes | We split the original dataset of 3,936 unique patient observations into training (validation) partitions with 3,736 (200) datums. For all experiments, the surrogate objective model fθ is a fully connected net with two hidden layers of size 2048 and Leaky Re LU activations. fθ takes as input a VAE-encoded latent space datum and returns the predicted objective function value as output. |
| Hardware Specification | Yes | We evaluate both GABO and GAGA against a number of pre-existing baseline algorithms on one internal cluster with 8 NVIDIA RTX A6000 GPUs. we are able to run Adaptive SCR with both Bayesian Optimization (BO) and Gradient Ascent (GA) using an experimental setup with one 24-core Intel Xeon CPU and one NVIDIA RTX A6000 GPU. |
| Software Dependencies | No | The paper describes software components like Adam optimizer and Leaky ReLU activations, but does not specify version numbers for general software dependencies like Python or PyTorch. |
| Experiment Setup | Yes | For all experiments, the surrogate objective model fθ is a fully connected net with two hidden layers of size 2048 and Leaky Re LU activations. We implement GABO using a quasi-expected improvement (q EI) acquisition function, iterative sampling budget of T = 32, sampling batch size of b = 64, and GAGA using a step size of η = 0.05, T = 128, and b = 16. We co-train the VAE and surrogate objective models together using an Adam optimizer with a learning rate of 3 10 4 for all tasks. |