Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
DesignX: Human-Competitive Algorithm Designer for Black-Box Optimization
Authors: Hongshu Guo, Zeyuan Ma, Yining Ma, Xinglin Zhang, Wei-Neng Chen, Yue-Jiao Gong
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we discuss the following research questions: RQ1: Can Design X automatically design human-competitive BBO optimizers that excel at both synthetic and realistic scenarios? ... Experiments Setup. The baselines in experiments include: ... The results in Table 1 reveal that: ... The results show that: ... Ablation Study (RQ3). ... We present the reversed normalized objective values ... in Figure 6. Detailed results for each problem set are provided in Appendix F.3. |
| Researcher Affiliation | Academia | Hongshu Guo1, Zeyuan Ma1, Yining Ma2, Xinglin Zhang1, Wei-Neng Chen1, Yue-Jiao Gong1, 1South China University of Technology 2Massachusetts Institute of Technology |
| Pseudocode | Yes | Algorithm 1: The pseudo code of the training of Design X Input: Training problem set Dtrain, Modular-EC M, initial Agent-1 model πϕ, Agent-2 actor πθ and critic vψ. Output: Well trained πϕ, πθ and vψ. // Training for Agent-1; Freeze πθ; for epoch = 1 to Epoch do... |
| Open Source Code | Yes | We provide Design X s Python project at https://github.com/Meta Evo/Design X. |
| Open Datasets | Yes | Large Scale Synthetic Problem Set. We construct a large scale synthetic problem set containing 12800 diverse problem instances for the ease of training generalizable Design X model. 32 representative basic problems are first collected from popular BBO benchmarks [61, 62], including Rastrigin, Schwefel, Rosenbrock, etc. ... Protein-Docking [79], a collection of 280 protein-docking instances; b) HPO-B [80], which comprises 86 ill-conditioning Auto ML instances; c) UAV [81], 56 diverse conflict-free UAV path planning scenarios... |
| Dataset Splits | Yes | We further randomly split them into a training problem set Dtrain (9600 instances) and a testing set Dtest (3200 instances). |
| Hardware Specification | Yes | All experiments are run on two Intel(R) Xeon(R) 6458Q CPUs with 488G RAM. |
| Software Dependencies | No | The paper mentions 'Transformer-based' models and 'GPT-2 blocks' but does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | In this paper, we set the embedding dimension h = 64 and the number of attention head k = 4 for both Agent-1 & 2. The number of blocks L is 1 for Agent-1 and 3 for Agent-2. The maximum number of modules M is 64 and the predefined maximum configuration size Nmax = 12. The training of both agents on Dtrain lasts for Epoch = 100 epochs with a fixed learning rate 0.0001. Agent-1 is trained with a batch size of 128. During the training of Agent-2, for a batch of 64 problems, PPO [64] method is used to update the policy and critic nets for kepoch = 3 times for every nstep = 10 rollout optimization steps. |