Minimax M-estimation under Adversarial Contamination
Authors: Sujay Bhatt, Guanhua Fang, Ping Li, Gennady Samorodnitsky
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5.2. Experimental Results In this section, we describe the experimental setup designed to evaluate the performance of the proposed algorithm against the existing baseline. Mukherjee et al. (2021) propose a successive elimination algorithm (SE-CBAI) for contaminated best arm identification sub-Gaussian setting, using a suitable trimmed mean estimator for robustness and confidence bounds adjusted to provide good sample complexity. Since SE-CBAI is proposed in the sub-Gaussian setting, we provide the performance comparison as shown in this setting, while allowing the contamination distribution to be selected from common models of noise. As the implementation details of SE-CBAI and technical details of another gap based algorithm (G-CBAI) for the asymptotic setting (δ 0) are not clear, we tuned the parameters that reflect the obtained performance in the paper and use that throughout for comparison. The hyper-parameters (Ω, τ, B, h, γ) in Algorithm 1 are tuned as follows: To compute the Catoni s estimator Eq. (4), for ε = 1, we need to calculate α, which depends on Ω (0, 1/Aη 4). Smaller Ωresults in smaller initial exploration, while increasing the magnitude of H(η) in Eq. (5) a quantity of interest from assumption A2. We choose Ω= 0.25 (1/Aη 4). The factor γ > 1 in Algorithm 1 that controls the exploration should be chosen barely greater than 1 for good performance, and we choose γ = 1.01. From Eq. (10), for ε < 1, we can choose h 0.5, and a τ (0, 2) that obtains a valid but large B (note that this affects the error bound in Eq. (10)). We take h = 0.5, τ = 1.2 and use B = 0.8. While the algorithm has more inputs than a typical successive elimination algorithm, it should be noted that tuning is straightforward here as we know the trade-offs. Results under different settings are summarized in Figure 1. Our method is uniformly better than SE-CBAI under various scenarios. |
| Researcher Affiliation | Collaboration | Sujay Bhatt, Guanhua Fang, Ping Li Gennady Samorodnitsky Cognitive Computing Lab School of ORIE Baidu Research Cornell University 10900 NE 8th St. Bellevue, WA 98004, USA 220 Frank T Rhodes Hall, Ithaca, NY 14853, USA {sujaybhatt.hr, fanggh2018, pingli98}@gmail.com gs18@cornell.edu |
| Pseudocode | Yes | Algorithm 1 Adversarial Elimination with Catoni (AECat) |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code or a direct link to a code repository for the methodology described. |
| Open Datasets | No | The paper mentions that 'The true/ uncontaminated distribution is taken to be Gaussian', and discusses experimental setup, but it does not specify any named publicly available datasets or provide concrete access information (link, DOI, citation) to a dataset used for training. |
| Dataset Splits | No | The paper does not explicitly provide specific dataset split information (percentages, sample counts, or citations to predefined splits) for training, validation, or testing. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific software dependency details, such as library names with version numbers, needed to replicate the experiment. |
| Experiment Setup | Yes | The hyper-parameters (Ω, τ, B, h, γ) in Algorithm 1 are tuned as follows: ... We choose Ω= 0.25 (1/Aη 4). The factor γ > 1 in Algorithm 1 that controls the exploration should be chosen barely greater than 1 for good performance, and we choose γ = 1.01. ... We take h = 0.5, τ = 1.2 and use B = 0.8. |