Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Fixed-Budget Best-Arm Identification in Structured Bandits
Authors: MohammadJavad Azizi, Branislav Kveton, Mohammad Ghavamzadeh
IJCAI 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using extensive experiments in Section 8, we show that our algorithm performs at least as well as a number of baselines, including Bayes Gap, Peace, and OD-Lin BAI. and 8 Experiments In this section, we compare GSE to several baselines including all linear FB BAI algorithms: Peace, Bayes Gap, and ODLin BAI. |
| Researcher Affiliation | Collaboration | Mohammad Javad Azizi1 , Branislav Kveton2 , Mohammad Ghavamzadeh3 1University of Southern California 2Amazon 3Google Research EMAIL, EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1 GSE: Generalized Successive Elimination and Algorithm 2 Frank-Wolfe G-optimal allocation (FWG) |
| Open Source Code | No | The paper does not provide an unambiguous statement or link for the open-source code of the methodology described. |
| Open Datasets | No | The paper describes experimental setups adopted from prior work (e.g., Soare et al. [2014], Xu et al. [2018], Tao et al. [2018], Yang and Tan [2021]) and how they generate their own data (e.g., 'generate i.i.d. arms sampled from the unit sphere', 'generate i.i.d. arms from uniform distribution'). However, it does not provide concrete access information (links, DOIs, repositories, or explicit citations to a public dataset resource) for a publicly available or open dataset that they directly use. |
| Dataset Splits | No | The paper does not provide specific dataset split information (exact percentages, sample counts, citations to predefined splits, or detailed splitting methodology) for training, validation, and test sets. Bandit problems typically do not use such splits as in supervised learning. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers) needed to replicate the experiments. |
| Experiment Setup | Yes | We set η = 2 in all experiments, as this value tends to perform well in successive elimination [Karnin et al., 2013]. For Lin Gap E, we evaluate the Greedy version (Lin Gap E-Greedy) and show its results only if it outperforms Lin Gap E. For Lin Gap EGreedy, see [Xu et al., 2018]. In each experiment, we fix K, B/K, or d; depending on the experiment to show the desired trend. Similar trends can be observed if we fix the other parameters and change these. For further detail of our choices of kernels for Bayes Gap and also our real-world data experiments, see Appendix E. |