Robust Outlier Arm Identification
Authors: Yinglun Zhu, Sumeet Katariya, Robert Nowak
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that our algorithms are both robust and about 5x sample efficient compared to state-of-the-art. |
| Researcher Affiliation | Collaboration | 1University of Wisconsin-Madison 2Amazon. Correspondence to: Yinglun Zhu <yinglun@cs.wisc.edu>. |
| Pseudocode | Yes | Algorithm 1 Construction of Confidence Intervals; Algorithm 2 ROAIElim; Algorithm 3 ROAILUCB |
| Open Source Code | Yes | Our code is publicly available (Zhu et al., 2020). URL https: //github.com/yinglunz/ROAI_ICML2020. |
| Open Datasets | Yes | We also compare the performance of all algorithms on the real-world Wine Quality dataset (Sathe & Aggarwal, 2016), which is widely used to compare outlier detection algorithms. |
| Dataset Splits | No | The paper describes generating synthetic data and simulating rewards from means of a real-world dataset but does not explicitly provide training, validation, and test splits for reproducibility in the traditional sense of fixed datasets. The multi-armed bandit setting involves adaptive sampling. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU or CPU models, processor types, or memory used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python 3.8, PyTorch 1.9). |
| Experiment Setup | Yes | We choose the arm configuration in Fig. 1(a) containing 15 normal arms (in blue) with fixed means equally distributed from 0 to 2, an outlier threshold θ ≈ 2.837, and 2 outlier arms (in orange) above the outlier threshold. The distance between the outlier arms is fixed at 0.2. We decrease θ from 0.6 to 0.2, and this changes the theoretical sample complexity. Note that the threshold does not change. The reward of each arm is normally distributed with standard deviation 0.5. (...) We generate 100 normal arm means from N(0.3, 0.075^2) and 5 outlier means from Unif(0.8, 1). We draw rewards of each arm from a Bernoulli distribution with respect to its mean. |