Learning Surrogates for Offline Black-Box Optimization via Gradient Matching
Authors: Minh Hoang, Azza Fadhel, Aryan Deshwal, Jana Doppa, Trong Nghia Hoang
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 3. Finally, we demonstrate the efficacy of MATCH-OPT on diverse real-world optimization problems from the designbench benchmark (Trabucco et al., 2022). Our results show that MATCH-OPT consistently shows improved optimization performance over existing baselines, and produces high-quality solutions with gradient search from diverse starting points (Section 5). and 5. Experiments This section describes the set of benchmark tasks used to evaluate and compare the performance of MATCH-OPT with those of other baselines (Section 5.1), the configurations of both our proposed algorithm and those baselines (Section 5.2), as well as their reported results (Section 5.3). |
| Researcher Affiliation | Academia | 1Lewis-Sigler Institute of Integrative Genomics, Princeton University, New Jersey, USA 2School of Electrical Engineering and Computer Science, Washington State University, Pullman, Washington, USA. |
| Pseudocode | Yes | Algorithm 1 MATCH-OPT: Black-Box Gradient Matching from Offline Training Data |
| Open Source Code | Yes | Our code is publicly available at https://github.com/azzafadhel/Match Opt. and Our implementation of the MATCH-OPT framework is released at https: //github.com/azzafadhel/Match Opt. |
| Open Datasets | Yes | Our empirical studies are conducted on six benchmark tasks from a diverse set engineering domains. Each task comprises a black-box target function and an offline training dataset, which is a small subset of a much larger dataset used to train the target function. The specifics of these datasets and their target functions are further provided in the design baseline package (Trabucco et al., 2022). |
| Dataset Splits | No | The paper discusses 'offline training data' and 'offline dataset' but does not explicitly provide details about training/validation/test splits or a separate validation set. |
| Hardware Specification | Yes | The computations were performed on a Ubuntu machine with a 3.73GHz AMD EPYC 7313 16-Core Processor (32 cores, 251 GB RAM) and two NVIDIA RTX A6000 GPUs. Each has 48 GB RAM. |
| Software Dependencies | No | The paper mentions software components like 'feed-forward neural network', 'Leaky ReLU function', and 'Adam optimizer', but does not provide specific version numbers for these software dependencies or libraries. |
| Experiment Setup | Yes | We used a feed-forward neural network with 4 layers (512 128 32 1) activated by the Leaky Re LU function as the surrogate model for MATCH-OPT. For each task, we trained the model using Adam optimizer with 1e-4 learning rate and a batch size of 128 for 200 epochs. During the evaluation, we employed gradient updates for 150 iterations uniformly across all the tasks. This evaluation procedure used an Adam optimizer with a 0.01 learning rate for all discrete tasks, and a 0.001 learning rate for all continuous tasks. |