Generative Pretraining for Black-Box Optimization
Authors: Satvik Mehul Mashkaria, Siddarth Krishnamoorthy, Aditya Grover
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we instantiate BONET using a causally masked Transformer (Radford et al., 2019) and evaluate it on Design-Bench (Trabucco et al., 2022), where we rank the best on average, outperforming stateof-the-art baselines. |
| Researcher Affiliation | Academia | 1Department of Computer Science, University of California, Los Angeles, US. Correspondence to: Siddarth Krishnamoorthy <siddarthk@cs.ucla.edu>. |
| Pseudocode | Yes | Algorithm 1 Black-box Optimization Networks (BONET) |
| Open Source Code | Yes | Our implementation of BONET can be found at https://github.com/siddarthk97/bonet. |
| Open Datasets | Yes | We evaluate our method on several real-world tasks in the Design-Bench (Trabucco et al., 2022) dataset. |
| Dataset Splits | No | The paper describes training and testing/evaluation phases, but does not explicitly mention a 'validation' dataset split or its specifics for hyperparameter tuning or model selection. |
| Hardware Specification | Yes | All training is done using 10 Intel(R) Xeon(R) CPU cores (E5-2698 v4443 @ 2.20GHz) and one NVIDIA Tesla V100 SXM2 GPU. |
| Software Dependencies | No | The paper mentions building upon 'min GPT' and 'Decision Transformer' codebases, but does not provide specific version numbers for programming languages, libraries, or other ancillary software dependencies used in the experiments. |
| Experiment Setup | Yes | The parameter details for all the tasks are summarized in the Table 5. Note that almost all of the parameters are same across all the Design-Bench tasks. Number of layers is higher for continuous tasks, as they are of higher dimensionalities. For all the tasks, we use a batch size of 128 and a fixed learning rate of 10 4 for 75 epochs. |