Truth-Tracking via Approval Voting: Size Matters
Authors: Tahar Allouche, Jérôme Lang, Florian Yger4768-4775
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We have conducted an experiment on three image annotation datasets; they conclude that rules based on our noise model outperform standard approval voting; the best performance is obtained by a variant of the Condorcet noise model. |
| Researcher Affiliation | Academia | Tahar Allouche,1 J erˆome Lang, 1 Florian Yger 1 1 LAMSADE, CNRS, PSL, Universit e Paris-Dauphine tahar.allouche@dauphine.eu, lang@lamsade.dauphine.fr, florian.yger@lamsade.fr |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | The code can be found at https://github.com/taharallouche/Truth Tracking-via-AV |
| Open Datasets | Yes | We took the three image annotation datasets, originally collected in (Shah, Zhou, and Peres 2015) for incentive-design purposes4, and used them to test our hypothesis and to assess the accuracy of different aggregation rules of interest. Accessible on the author s webpage: https://cs.cmu.edu/ nihars/data/data approval.zip |
| Dataset Splits | No | The paper describes the datasets (Animal task: 16 images/questions and 6 alternatives. Texture task: 16 images/questions and 6 alternatives. Language task: 25 images/questions and 8 alternatives.) and states 'For each task, we took 25 batches for each different number of voters, and applied the aforementioned rules.' but does not provide specific training, validation, or test dataset splits or percentages. |
| Hardware Specification | No | The paper does not contain any specific details about the hardware used for running the experiments. |
| Software Dependencies | No | The paper provides a link to a GitHub repository for its code, implying software is involved, but does not list any specific software dependencies with version numbers. |
| Experiment Setup | No | The paper describes the aggregation methods and some parameters (like precision parameter pz i and weights wj), and mentions taking '25 batches for each different number of voters', but it does not provide specific experimental setup details such as hyperparameter values (e.g., learning rate, batch size) or other system-level training settings. |