Boosting Offline Optimizers with Surrogate Sensitivity
Authors: Manh Cuong Dao, Phi Le Nguyen, Thao Nguyen Truong, Trong Nghia Hoang
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This development is both orthogonal and synergistic to prior research on offline optimization, which is demonstrated in our extensive experiment benchmark. ... We demonstrate the empirical efficiency of the proposed regularization method on a wide variety of benchmark problems, which shows consistently that its synergistic performance boost on existing offline optimizers is significant. |
| Researcher Affiliation | Academia | 1School of Information and Communications Technology, Hanoi University of Science and Technology, Hanoi, Vietnam 2National Institute of Advanced Industrial Science and Technology, Tokyo, Japan 3School of Electrical Engineering and Computer Science, Washington State University, Washington, USA. |
| Pseudocode | Yes | The pseudocode of our boosting framework for offline optimizers via surrogate sensitivity (BOSS) is detailed in Algorithm 1. |
| Open Source Code | Yes | The code for reproducing our results is at https: //github.com/daomanhcuonghust/BOSS |
| Open Datasets | Yes | Our empirical evaluations are conducted on 6 real-world tasks from Design-Bench (Trabucco et al., 2022), with both discrete and continuous domains. ... The discrete tasks include TF-Bind-8, TF-Bind-10, and Ch EMBL where TF-Bind-8 and TF-Bind-10 (Barrera et al., 2016)... and Ch EMBL is derived from a drug property database (Gaulton et al., 2012)... The continuous tasks include Ant Morphology (Brockman et al., 2016), D Kitty Morphology (Ahn et al., 2020), and Superconductors (Brookes et al., 2019). |
| Dataset Splits | No | The paper mentions using "offline data" for training and a "test dataset" for evaluation but does not specify a distinct validation set with details on its split (e.g., percentages or counts) or how it's used in a formal train/validation/test partitioning. |
| Hardware Specification | No | The paper does not explicitly mention specific hardware components such as GPU models (e.g., NVIDIA A100, RTX 3090) or CPU models (e.g., Intel Xeon, AMD Ryzen) used for running the experiments. |
| Software Dependencies | No | The paper specifies the architecture of the neural network used for Φ ("Φ is a neural network with one hidden layer comprising two hidden units and one output layer.") but does not list any software dependencies (e.g., libraries, frameworks) with specific version numbers. |
| Experiment Setup | Yes | Our proposed regularizer BOSS also has additional hyperparameters (α, ω) as highlighted previously in Section 3.2. In particular, ω = (ωµ, ω2 σ) is a tuple of learnable parameters that define the (adversarial) perturbation distribution N(ωµ1, ω2 σI)... We have conducted ablation studies in Section 5.3 to find the most appropriate bounds for these parameters, which appear to be [ 10 3, 10 3] for [ωµl, ωµu] and [10 5, 10 2] for [ωσl, ωσu]. We use the above bounds in all experiments, in which ωµ and ω2 σ are initialized to 0 and 10 3, respectively. Likewise, for the sensitivity threshold, our ablation studies observe the impact of several values of α on the performance and find that α = 0.1 is the best universal value for BOSS. In addition, we set the weight of the regularizer λ to 10 3, the no. m of perturbation sample per iteration to 100 and the learning rates ηω = 10 2, ηϕ = 10 3. |