Combating Collusion Rings Is Hard but Possible
Authors: Niclas Boehmer, Robert Bredereck, André Nichterlein4843-4850
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we compare the weight of review assignments computed using different methods and analyze the occurrences of review cycles. For this, we use a dataset from the 2018 International Conference on Learning Representations (ICLR 18) prepared by Xu et al. (2019). |
| Researcher Affiliation | Academia | 1 Technische Universit at Berlin, Faculty IV, Algorithmics and Computational Complexity, Berlin, Germany 2 Humboldt-Universit at zu Berlin, Institut f ur Informatik, Algorithm Engineering, Berlin, Germany |
| Pseudocode | Yes | Algorithm 1: A greedy algorithm computing a dpaperdpaper-valid completely cycle-free assignment E . And Algorithm 2: Greedy algorithm to compute a creviewerdpaper-valid z-cycle-free review assignment E . |
| Open Source Code | Yes | The code for our experiments is available at github.com/nboehmer/Combating-Collusion-Rings-is-Hard-but-Possible. |
| Open Datasets | Yes | For this, we use a dataset from the 2018 International Conference on Learning Representations (ICLR 18) prepared by Xu et al. (2019). |
| Dataset Splits | No | From the dataset of Xu et al. (2019), we created multiple instances of WEIGHTED CYCLE-FREE REVIEWING as follows. Given a number n P of papers and a ratio r AP of the numbers of agents and papers, we sample a subset of n P of the 911 ICLR 18 papers and set this as our set of papers. Subsequently, we compute the set of all authors of one of these papers and sample a subset of r AP n P authors and set this as our set of agents. The paper does not specify train/validation/test dataset splits. |
| Hardware Specification | No | The paper states that "(I)LPs were solved using Gurobi Optimization, LLC (2021)" and provides timing comparisons, but it does not specify the hardware (e.g., CPU, GPU, memory) on which these experiments were run. |
| Software Dependencies | Yes | We solved all (I)LPs using Gurobi Optimization, LLC (2021). |
| Experiment Setup | Yes | As done in other papers using the same dataset, we focus on the case with dpaper = 3 and creviewer = 6, i.e., every paper needs exactly three reviews and each agent can review at most six papers (Xu et al. 2019; Jecmen et al. 2020). And Given a number n P of papers and a ratio r AP of the numbers of agents and papers, we sample a subset of n P of the 911 ICLR 18 papers and set this as our set of papers. Subsequently, we compute the set of all authors of one of these papers and sample a subset of r AP n P authors and set this as our set of agents. |