Bidirectional Learning for Offline Infinite-width Model-based Optimization
Authors: Can Chen, Yingxueff Zhang, Jie Fu, Xue (Steve) Liu, Mark Coates
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on various tasks verify the effectiveness of BDI. |
| Researcher Affiliation | Collaboration | 1Mc Gill University, 2 Huawei Noah s Ark Lab, 3 Beijing Academy of Artificial Intelligence |
| Pseudocode | No | The paper describes the method using mathematical equations and text but does not include a clearly labeled pseudocode or algorithm block. |
| Open Source Code | No | The code is available here. (In abstract). We include our code in the supplemental material as a new asset and we will release this new asset upon paper acceptance. |
| Open Datasets | Yes | To evaluate the effectiveness of BDI, we adopt the commonly used design-bench, including both continuous and discrete tasks, and the evaluation protocol as in the prior work [8]. |
| Dataset Splits | Yes | We follow the training settings of [8] for all comparison methods if not specified. |
| Hardware Specification | Yes | All Jax experiments are run on multiple CPUs within a cluster and all Pytorch experiments are run on one V100 GPU. |
| Software Dependencies | No | We use the NTK library [29] build on Jax [30] to conduct kernel-based experiments and use Pytorch [31] for other experiments. |
| Experiment Setup | Yes | We set the regularization β as 1e 6 following [16]. We set the predefined target score yh as a constant 10 across all tasks, which proves to be effective and robust in all cases. We set α as 1e 3 for all continuous tasks and as 0.0 for all discrete tasks, and set the number of iterations T to 200 in all experiments. We adopt a 6-layer MLP (Multi Layer Perceptron) followed by Re LU for all gradient updating methods and the hidden size is set as 2048. We optimize L(Xh) with the Adam optimizer [21] with a 1e 1 learning rate for discrete tasks and a 1e 3 learning rate for continuous tasks. |