A/B/n Testing with Control in the Presence of Subpopulations
Authors: Yoan Russac, Christina Katsimerou, Dennis Bohle, Olivier Cappé, Aurélien Garivier, Wouter M. Koolen
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We illustrate the efficiency of the proposed strategy with numerical simulations on synthetic and real data collected from an A/B/n experiment. |
| Researcher Affiliation | Collaboration | Yoan Russac CNRS, Inria, ENS Université PSL yoan.russac@ens.fr Christina Katsimerou Booking.com christina.katsimerou@booking.com Dennis Bohle Booking.com dennis.bohle@booking.com Olivier Cappé CNRS, Inria, ENS Université PSL olivier.cappe@cnrs.fr Aurélien Garivier UMPA, CNRS Inria, ENS Lyon aurelien.garivier@ens-lyon.fr Wouter M. Koolen Centrum Wiskunde & Informatica wmkoolen@cwi.nl |
| Pseudocode | No | The paper describes the algorithm steps in text but does not provide a formal pseudocode block or algorithm figure. |
| Open Source Code | Yes | Code at https://gitlab.com/ckatsimerou/abc_s_public |
| Open Datasets | No | The paper mentions using "real data collected from an A/B/n experiment" and synthetically generated data, but does not provide access information (link, DOI, citation) for a publicly available dataset. |
| Dataset Splits | No | The paper does not explicitly provide training/validation/test dataset splits needed to reproduce the experiment. |
| Hardware Specification | No | The paper does not specify any hardware used for running the experiments (e.g., GPU models, CPU types, or cloud instance details). |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., library names with versions). |
| Experiment Setup | Yes | In our second experiment 1, we generated 3000 Bernoulli bandit instances with K = 2 and a random number of subpopulations J between 2 and 10. Each subpopulation-arm s mean µa,i is drawn uniformly at random from [0, 1], and the subpopulation frequency vector α is drawn from a Dirichlet(10) distribution. |