Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Learning the Valuations of a $k$-demand Agent
Authors: Hanrui Zhang, Vincent Conitzer
ICML 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In Section 4, we experimentally evaluate the performance of ERM algorithms. |
| Researcher Affiliation | Academia | 1Department of Computer Science, Duke University, Durham, USA. Correspondence to: Hanrui Zhang <EMAIL>, Vincent Conitzer <EMAIL>. |
| Pseudocode | Yes | Algorithm 1 Biased Binary Search |
| Open Source Code | No | The paper does not contain any statement about making its source code publicly available, nor does it provide a link to a code repository. |
| Open Datasets | No | We draw the ground truth value vector uniformly at random from the unit hypercube [0, 1]n, and for each sample, we draw the price vector uniformly at random from [−1, 0]n. The paper generates its own data for experiments and does not use a pre-existing public dataset with concrete access information. |
| Dataset Splits | No | The paper mentions training and testing but does not explicitly describe a validation dataset split. |
| Hardware Specification | No | The paper does not provide any specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions using an 'LP solver' but does not provide specific version numbers for it or any other software dependencies. |
| Experiment Setup | Yes | We implement the ERM learner by solving the system in Proposition 2 using an LP solver, where the objective is to maximize the minimum margin. We draw the ground truth value vector uniformly at random from the unit hypercube [0, 1]n, and for each sample, we draw the price vector uniformly at random from [−1, 0]n. To study the performance of ERM for different values of k, we fix the number of items to be n = 50, and examine the accuracy of the ERM learner for k ∈ {5, 10, 15, 20, 25} respectively. To study the performance of ERM for different values of n, we fix the agent to be unit-demand (i.e., k = 1), and calculate the accuracy of the ERM learner for n ∈ {20, 40, 60, 80, 100} respectively. In both experiments, we let the size of the training set grow, and plot the empirical error rate for each size of the training set ℓ ∈ {50, 100, 150, 200, 250, 300, 350, 400, 450, 500}. |