Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Projective Preferential Bayesian Optimization
Authors: Petrus Mikkola, Milica Todorović, Jari Järvi, Patrick Rinke, Samuel Kaski
ICML 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section we demonstrate the effciency of the PPBO method in high-dimensional spaces, and experiment with various acquisition strategies in numerical experiments on simulated functions. (Section 4); In this section we demonstrate the capability of PPBO to correctly and effciently encode user preferences from pro jective preferential feedback. We consider a material science problem... (Section 5) |
| Researcher Affiliation | Academia | 1Helsinki Institute for Information Technology HIIT, De partment of Computer Science, Aalto University, Espoo, Fin land 2Department of Applied Physics, Aalto University, Espoo, Finland 3The University of Manchester, UK. |
| Pseudocode | Yes | Algorithm 1 Approximate EIn(ξ, x) |
| Open Source Code | Yes | Source code is available at https://github.com/Aalto PML/PPBO. |
| Open Datasets | Yes | For f we consider four different test functions: Six-hump camel2D, Hartmann6D, Levy10D and Ackley20D.3 (Footnote 3 points to 'https://www.sfu.ca/~ssurjano/optimization.html' which makes these standard benchmark functions openly accessible.) |
| Dataset Splits | No | The paper uses benchmark test functions and a user experiment, but does not provide specific details on train/validation/test dataset splits with percentages or counts for reproducibility. |
| Hardware Specification | Yes | All experiments of each test function were run on a computing infrastructure of 24x Xeon Gold 6148 2.40GHz cores and 72GB RAM. (Footnote 4, Section 4) |
| Software Dependencies | No | The paper mentions 'quantum mechanical atomistic simulation code FHI-aims (Blum et al., 2009)' but does not provide specific version numbers for it or any other software dependencies needed to replicate the experiment. |
| Experiment Setup | Yes | We consider a total budget of 100 queries. The ith-initial query corresponds to ξ = ei, that is, to the ith-coordinate projection, and the reference vector x is uniformly random. (Section 4); The total number of queries was 24, of which 6 were initial queries. The ith-initial query corresponded to ξ = ei, that is, to the ith-coordinate projection. The initial values for the reference coordinate vector x were fxed to the same value across all user sessions. For acquisition, we used the expected improvement by projective preferential query. (Section 5) |