Peer-Prediction in the Presence of Outcome Dependent Lying Incentives
Authors: Naman Goel, Aris Filos-Ratsikas, Boi Faltings
IJCAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we evaluate the savings of PTSC experimentally on two real-world datasets, described below. |
| Researcher Affiliation | Academia | 1Swiss Federal Institute of Technology, Lausanne (EPFL) 2University of Liverpool, UK |
| Pseudocode | No | The paper describes the PTSC mechanism and other concepts verbally and mathematically, but does not provide pseudocode or a clearly labeled algorithm block. |
| Open Source Code | No | The paper does not include an explicit statement about providing open-source code for the methodology described, nor does it provide a direct link to a code repository. |
| Open Datasets | Yes | We conducted experiments on the dataset3 of [Zheng et al., 2014], which contains real-world Quality of Service evaluation results from 339 trusted agents on 5,825 web services. ... Dataset is available at http://wsdream.github.io. |
| Dataset Splits | No | The paper describes using a dataset and sampling observations for approximation, but it does not specify explicit training, validation, and test dataset splits (e.g., percentages, counts, or references to predefined splits) used in their experiments. |
| Hardware Specification | No | The paper does not provide specific details regarding the hardware (e.g., CPU, GPU models, or cloud configurations) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependency details, such as library names with version numbers, needed to replicate the experiment. |
| Experiment Setup | Yes | We fix a constant refund amount R in our simulations; since we will only discuss the relative saving, the actual choice of R is not important here. We vary the number of agents that are asked to report their observations for a service, by randomly selecting a subset of the agents from the dataset for every web-service. |