Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Gradient-Based Optimization for Bayesian Preference Elicitation
Authors: Ivan Vendrov, Tyler Lu, Qingqing Huang, Craig Boutilier10292-10301
AAAI 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that our gradientbased EVOI technique achieves state-of-the-art performance across several domains while scaling to large item spaces. |
| Researcher Affiliation | Industry | Ivan Vendrov, Tyler Lu, Qingqing Huang, Craig Boutilier Google Research, Mountain View, California EMAIL |
| Pseudocode | Yes | Algorithm 1 Deep Retr Uniq. Inputs: optimized X and U |
| Open Source Code | No | The paper mentions using "Tensor Flow, Py Torch" as computational frameworks but does not provide any statement about releasing their own source code or a link to a repository. |
| Open Datasets | Yes | Using the Movie Lens 100-K dataset, we train user and movie embeddings with dimension d = 10. ... We use the Movie Lens-20M (Harper and Konstan 2015) dataset and represent each movie with 100 binary attributes from the Tag Genome (Vig, Sen, and Riedl 2012). |
| Dataset Splits | No | The paper uses datasets like Movie Lens 100-K and Goodreads, and describes running elicitation trials with random selections of items and user embeddings, but it does not provide specific train/validation/test dataset split percentages or counts. |
| Hardware Specification | Yes | We benchmark the algorithmic runtimes on a workstation with a 12-core Intel Xeon E5-1650 CPU at 3.6GHz, and 64GB of RAM. |
| Software Dependencies | No | The paper mentions using "Tensor Flow, Py Torch" as computational frameworks but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | We initialize query embeddings to random uniform values in [0, 1]100, then run gradient ascent on Eq. 7 for 100 steps, initializing the regularization weight λ at 0.01 and multiplying λ by 1.1 each iteration. |