Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Mixture Martingales Revisited with Applications to Sequential Tests and Confidence Intervals
Authors: Emilie Kaufmann, Wouter M. Koolen
JMLR 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We illustrate this empirically in Figure 4 for a Bernoulli bandit model with M arms with mean 0.1 and 4 more arms with means 0.2, 0.3, 0.4, 0.5 (thus K = M + 4), for different values of M. We consider the use of a Box weight vector that is uniform on the singletons (π(1) = 1), a weight vector supported on the whole set of arms (π(K) = 1) and a weight vector that is uniform over subset sizes (π(k) = 1/K). For each value of M, data is collected using uniform sampling and we set δ = 10^-10 to focus on the high confidence regime. |
| Researcher Affiliation | Academia | Emilie Kaufmann Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRISt AL, F-59000 Lille, France EMAIL; Wouter M. Koolen Centrum Wiskunde & Informatica, Science Park 123, Amsterdam, Netherlands EMAIL |
| Pseudocode | No | The paper describes algorithms and methods verbally, such as the 'Tracking rule' and 'GLR stopping rule', but does not provide any explicitly labeled pseudocode blocks or algorithms in a structured, code-like format. |
| Open Source Code | No | The paper does not contain any explicit statement about releasing source code for the described methodology, nor does it provide a link to a code repository. |
| Open Datasets | No | The paper discusses various types of distributions (e.g., Gaussian, Bernoulli, Gamma) for multi-armed bandit models and uses a 'Bernoulli bandit model' for empirical illustration. However, it does not provide access information (link, DOI, specific citation to an external dataset with authors/year) for any publicly available or open dataset. The empirical illustration uses simulated data based on specific mean values, not a pre-existing dataset. |
| Dataset Splits | No | The paper does not use external datasets and therefore does not provide any training/test/validation dataset split information. |
| Hardware Specification | No | The paper does not mention any specific hardware (e.g., GPU models, CPU types, or cloud computing specifications) used for running its experiments or simulations. |
| Software Dependencies | Yes | For both Gaussian and Bernoulli (and possibly more) we can write the objective as a Disciplined Convex Program and solve it efficiently with e.g. CVX (Grant and Boyd, 2017). CVX: Matlab software for disciplined convex programming, version 2.1. http://cvxr.com/cvx, Mar. 2017. |
| Experiment Setup | Yes | We illustrate this empirically in Figure 4 for a Bernoulli bandit model with M arms with mean 0.1 and 4 more arms with means 0.2, 0.3, 0.4, 0.5 (thus K = M + 4), for different values of M. We consider the use of a Box weight vector that is uniform on the singletons (π(1) = 1), a weight vector supported on the whole set of arms (π(K) = 1) and a weight vector that is uniform over subset sizes (π(k) = 1/K). For each value of M, data is collected using uniform sampling and we set δ = 10^-10 to focus on the high confidence regime. |