Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Online Selection of Diverse Committees
Authors: Virginie Do, Jamal Atif, Jérôme Lang, Nicolas Usunier
IJCAI 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We study three methods, theoretically and experimentally: a greedy algorithm that includes volunteers as long as proportionality is not violated; a non-adaptive method that includes a volunteer with a probability depending only on their features, assuming that the joint feature distribution in the volunteer pool is known; and a reinforcement learning based approach when this distribution is not known a priori but learnt online. For each, we study bounds for expected quality and sample complexity, and perform experiments using real data from the UK Citizens Assembly on Brexit. The outline of the paper is as follows. We discuss related work in Section 2, define the problem in Section 3, define and study our three strategies in Sections 3.2, 4 and 5, analyse our experiments in Section 6 and conclude in Section 7. |
| Researcher Affiliation | Collaboration | Virginie Do1,2 , Jamal Atif1 , J erˆome Lang1 and Nicolas Usunier2 1LAMSADE, Universit e PSL, Universit e Paris-Dauphine, CNRS, France 2Facebook AI Research EMAIL, EMAIL, EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1: RL-CMDP algorithm. |
| Open Source Code | No | Full version available at https://arxiv.org/abs/2105.09295. |
| Open Datasets | Yes | To answer these questions, we use summary data from the 2017 Citizens Assembly on Brexit. The participants were recruited in an offline manner: volunteers could express interest in a survey, and then 53 citizens were drawn from the pool of volunteers using stratified sampling, in order to construct an assembly that reflects the diversity of the UK electorate. We use summary statistics published in the report [Renwick et al., 2017] to simulate an online recruitment process. |
| Dataset Splits | No | The paper describes simulating an online recruitment process and averaging results over 50 simulations, but it does not specify explicit training/validation/test splits of a dataset in the conventional machine learning sense, as it focuses on an online selection problem. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory) used for running experiments were mentioned in the paper. |
| Software Dependencies | No | No specific software dependencies with version numbers were mentioned in the paper (e.g., Python 3.8, PyTorch 1.9, CPLEX 12.4). |
| Experiment Setup | Yes | We study Greedy with tolerance ϵ = 0.02, 0.05. We run experiments for K = 50, 100, 150, 250, 500, 1000, averaged over 50 simulations. |