Avoiding Imposters and Delinquents: Adversarial Crowdsourcing and Peer Prediction
Authors: Jacob Steinhardt, Gregory Valiant, Moses Charikar
NeurIPS 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | Our technical tools draw on semidefinite programming methods for matrix completion... The proof of Theorem 2 can be split into two parts: analyzing Algorithm 1 (Section 4), and analyzing Algorithm 2 (Section 5). Proving Proposition 1 involves two major steps... Lemma 1. Let m n. Suppose that Assumption 1 holds. Lemma 2. Let m n. Then there is a k = O log3(2/δ) n such that... Lemma 3. Let i be the row selected in Algorithm 2. Suppose that r satisfies Assumption 1. Lemma 4. The output T of Algorithm 4 satisfies E[T] = T0, T 0 βm. |
| Researcher Affiliation | Academia | Jacob Steinhardt Stanford University Gregory Valiant Stanford University Moses Charikar Stanford University |
| Pseudocode | Yes | Algorithm 1 Algorithm for recovering β-quantile matrix M using (unreliable) ratings A. Algorithm 2 Algorithm for recovering an accurate β-quantile T from the β-quantile matrix M. Algorithm 3 Algorithm for obtaining (unreliable) ratings matrix A and noisy ratings r, r . Algorithm 4 Randomized rounding algorithm. |
| Open Source Code | No | No statement regarding the release of open-source code for the described methodology was found. No links to repositories or mentions of supplementary materials containing code were present. |
| Open Datasets | No | The paper is theoretical and describes a model with 'n workers' and 'm items' but does not specify or use any particular dataset for empirical evaluation, nor does it provide access information for any dataset. |
| Dataset Splits | No | This paper is theoretical and does not involve empirical data or experiments, therefore there is no mention of dataset splits for training, validation, or testing. |
| Hardware Specification | No | This paper is theoretical and does not describe any empirical experiments, thus no hardware specifications for running experiments are mentioned. |
| Software Dependencies | No | This paper is theoretical and does not describe any empirical experiments or implementations, thus no software dependencies with version numbers are mentioned. |
| Experiment Setup | No | This paper is theoretical and does not describe any empirical experiments, thus no experimental setup details like hyperparameters or training settings are provided. |