Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Information Gathering With Peers: Submodular Optimization With Peer-Prediction Constraints
Authors: Goran Radanovic, Adish Singla, Andreas Krause, Boi Faltings
AAAI 2018 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we evaluate our methods using a realistic crowd sensing testbed. ... We perform two different tests. In the first test, we vary the total available budget B from 5 to 25 at steps of size 5. At the same time, we keep the minimal expected payment to τmin = 0.5. As we can see from Figure 2a, as the total budget increases, all the methods perform better. ... The results are given in Figure 2b. |
| Researcher Affiliation | Academia | Goran Radanovic Harvard University Cambridge, USA EMAIL Adish Singla MPI-SWS Saarbr ucken, Germany EMAIL Andreas Krause ETH Zurich Zurich, Switzerland EMAIL Boi Faltings EPFL Lausanne, Switzerland boi.faltings@epfl.ch |
| Pseudocode | Yes | Algorithm 1: Algorithm PPCGREEDY... Algorithm 2: Algorithm PPCGREEDYITER |
| Open Source Code | No | The paper does not include any explicit statements or links indicating the release of open-source code for the described methodology. |
| Open Datasets | Yes | We use a crowd sensing test-bed of Singla (2017), constructed from real measurements of CO2 and user locations across an urban area. ... obtained using a publicly available data (Open Street Map5), from which we randomly selected 300 locations from an area in the center of New York City. ... inferred from a publicly accessible dataset (Strava6) that contains the mobility patterns of cyclists for a period of 6 days. |
| Dataset Splits | No | The paper describes how the dataset was collected and sampled but does not specify a training, validation, or test split for the data used in experiments. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments, such as CPU/GPU models or cloud instance types. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers, such as programming languages or libraries. |
| Experiment Setup | Yes | In the first test, we vary the total available budget B from 5 to 25 at steps of size 5. At the same time, we keep the minimal expected payment to τmin = 0.5. ... In the second test, we vary the minimal expected payment τmin, which now takes values in {0.1, 0.25, 0.5, 0.75, 0.9}. Budget B is set to 15. |