Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Finding All $\epsilon$-Good Arms in Stochastic Bandits
Authors: Blake Mason, Lalit Jain, Ardhendu Tripathy, Robert Nowak
NeurIPS 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We introduce two algorithms to overcome these and demonstrate their great empirical performance on a large-scale crowd-sourced dataset of 2.2M ratings collected by the New Yorker Caption Contest as well as a dataset testing hundreds of possible cancer drugs. |
| Researcher Affiliation | Academia | Blake Mason University of Wisconsin Madison, WI 53706 EMAIL Lalit Jain University of Washington Seattle, WA 98115 EMAIL Ardhendu Tripathy University of Wisconsin Madison, WI 53706 EMAIL Robert Nowak University of Wisconsin Madison, WI 53706 EMAIL |
| Pseudocode | Yes | Algorithm 1 (ST)2: Sample the Threshold, Split the Threshold ... Algorithm 4.1: additive FAREAST with γ = 0 |
| Open Source Code | Yes | Implementations of all algorithms and baselines used in this paper are available on Git Hub. |
| Open Datasets | No | The paper mentions using a "large-scale crowd-sourced dataset of 2.2M ratings collected by the New Yorker Caption Contest" and a "dataset [27] of 189 inhibitors". While it cites a paper for the cancer drug dataset, it does not provide a direct link, DOI, or repository access information for either dataset. |
| Dataset Splits | No | The paper does not specify explicit training, validation, or test dataset splits (e.g., percentages or sample counts). |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments, such as CPU/GPU models, memory, or cloud instance types. |
| Software Dependencies | No | The paper does not list any specific software dependencies with version numbers (e.g., programming languages, libraries, or frameworks). |
| Experiment Setup | Yes | In the first example on the left, δ = 0.1, = β = 0.05. ... for δ = 0.01... We set = 0.1 and focus on the multiplicative setting... In this experiment, we use the multiplicative case of ALLwith = 0.8 and δ = 0.001. |