Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Budgeted Optimization with Constrained Experiments
Authors: Javad Azimi, Xiaoli Fern, Alan Fern
JAIR 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our proposed methods for both settings using synthetic and real functions. The experimental results demonstrate the efficacy of the proposed methods. [...] 6. Experimental Results |
| Researcher Affiliation | Collaboration | Javad Azimi EMAIL Microsoft, Sunnyvale, CA, USA; Xiaoli Z. Fern EMAIL School of EECS, Oregon State University; Alan Fern EMAIL School of EECS, Oregon State University |
| Pseudocode | Yes | Algorithm 1 The Greedy Non-Sequential Algorithm [...] Algorithm 2 Accelerated Greedy Algorithm |
| Open Source Code | No | No explicit statement or link for the open-sourcing of the code described in this paper was found. |
| Open Datasets | Yes | The first three functions: Cosines, Rosenbrock, and Discontinuous are benchmarks that have been widely used in previous studies on stochastic optimization (Anderson, Moore, & Cohn, 2000; Brunato, Battiti, & Pasupuleti, 2006; Azimi et al., 2010). [...] For the former we utilize data collected as part of a study on biosolar hydrogen production (Burrows, Wong, Fern, Chaplen, & Ely, 2009) |
| Dataset Splits | No | The paper describes using test functions and a set of initial random points for policy evaluation, but does not specify conventional train/test/validation dataset splits for models or evaluation. |
| Hardware Specification | No | The paper mentions 'un-optimized matlab implementation' and describes run times, but does not provide specific hardware details (e.g., CPU/GPU models, memory, or machine specifications) used for running experiments. |
| Software Dependencies | No | The paper mentions using 'Gaussian process' and an 'un-optimized matlab implementation' but does not provide specific version numbers for any software, libraries, or solvers used to replicate the experiment. |
| Experiment Setup | Yes | In this paper we set κ = 0.02 and signal variance σf = y2max [...] We evaluate our proposed approaches considering three different slope values; slope = 0.1, 0.15, 0.30. [...] we divide each input dimension into 100 equal-length subintervals. [...] Each run starts with n = 5 randomly selected initial points [...] we fixed the total budget to B = 15 and examine the effect of the cost-model slope parameter over values 0.1, 0.15 and 0.3. In later experiments, we will consider larger budgets. |