Peer Prediction for Learning Agents
Authors: Shi Feng, Fang-Yi Yu, Yiling Chen
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Simulation of several algorithms in this family as well as the ϵ-greedy algorithm, which is outside of this family, shows convergence to the truthful strategy in the CA mechanism. [...] We simulate the CA mechanism with various learning algorithms: the Hedge algorithms, follow the perturbed leader, follow the leader, and ϵ-greedy, and repeat the process 400 times with 800 rounds on each algorithm each time. |
| Researcher Affiliation | Academia | Shi Feng Institute for Interdisciplinary Information Sciences, Tsinghua University Beijing, China [...] Fang-Yi Yu Department of Computer Science, George Mason University Fairfax, VA, USA [...] Yiling Chen John A. Paulson School of Engineering and Applied Sciences, Harvard University Cambridge, MA, USA |
| Pseudocode | No | The paper describes algorithms and processes mathematically and textually, but it does not include any formal pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement about releasing source code or a link to a code repository for the methodology described. |
| Open Datasets | No | The paper describes a simulation setup for signal distribution ("PX,Y (0, 0) = PX,Y (1, 1) = 0.4, PX,Y (1, 0) = PX,Y (0, 1) = 0.2") but does not use or provide access information for a publicly available or open dataset. |
| Dataset Splits | No | The paper conducts simulations but does not describe the use of dataset splits for training, validation, or testing in the conventional machine learning sense. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU/CPU models, memory) used to run the simulations. |
| Software Dependencies | No | The paper discusses various learning algorithms (Hedge, FPL, FTL, ϵ-greedy) but does not specify software dependencies or their version numbers used in the simulations. |
| Experiment Setup | Yes | We simulate the CA mechanism with various learning algorithms: the Hedge algorithms, follow the perturbed leader, follow the leader, and ϵ-greedy, and repeat the process 400 times with 800 rounds on each algorithm each time. [...] In our simulations, we use the following private signal distribution that satisfies assumption 2.2: PX,Y (0, 0) = PX,Y (1, 1) = 0.4, PX,Y (1, 0) = PX,Y (0, 1) = 0.2. [...] FPL algorithm given a noise distribution N on scalars, f FPL i (R1, R2, R3, R4) = Pr n Ri + pi = maxj [4]{Rj + pj} pj iid N, j [4] o for i [4]. We consider FPL1, FPL4, and FPL8. Hedge algorithm 1 choose i with probability proportional to 3Ri/2 that is an implementation by choosing ϵ = 0.5 for the multiplicative weights algorithm of Arora et al. [2]. Hedge algorithm 2 chooses i with probability proportional to e Ri that is an implementation by choosing β = 1 of exponentially weighted averaged forecaster introduced by Freund and Schapire [10]. Finally, ϵ-greedy algorithm uses time varying ϵ = 1 (t+1)2 at round t. |