Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Nearly Optimal Catoni’s M-estimator for Infinite Variance
Authors: Sujay Bhatt, Guanhua Fang, Ping Li, Gennady Samorodnitsky
ICML 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically validate the performance improvement in using our proposed algorithm based on Catoni’s estimator for ε ∈ (0, 1] by comparing it with the state of the art best arm identification algorithms proposed in Yu et al. (2018). (...) A significant improvement in performance using our algorithms is observed in Figure 1, more so in case of smaller ε, indicating the tightness for smaller ε. Table 1 shows the performance of Algorithm 3 for K from 10 to 1000. |
| Researcher Affiliation | Collaboration | Sujay Bhatt, Guanhua Fang, Ping Li Gennady Samorodnitsky Cognitive Computing Lab School of ORIE Baidu Research Cornell University 10900 NE 8th St. Bellevue, WA 98004, USA 220 Frank T Rhodes Hall, Ithaca, NY 14853, USA EMAIL EMAIL |
| Pseudocode | Yes | Algorithm 1 Lepskii Moment Adaptation (LMA) (...) Algorithm 2 Iterative Elimination with Catoni (...) Algorithm 3 Phase-based Iterative Elimination with Catoni (...) Algorithm 4 Adaptive Elimination with Catoni (...) Algorithm 5 Restricted Differential Privacy for Heavy-tailed Data |
| Open Source Code | No | The paper does not contain an explicit statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | No | The experimental setup is as follows: the rewards for all arms in all experiments are generated from the heavy tailed Student’s-t distribution. The shifted mean of each arm is given using the following rule: µ(i) = 2 − (i − 1/K)0.6 for i = 2, ..., K and µ(1) = 2. The number of degrees of freedom for the first two figures from the left is ν = 3 which corresponds to ε = 1 and υε = 3. The third figure uses ν = 2 which has infinite variance and for ε = 0.85, an upper bound is υε = 50. |
| Dataset Splits | No | The paper describes experiments in a multi-armed bandit setting with sequentially generated data, rather than using predefined training/validation/test splits from a fixed dataset. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware (e.g., CPU, GPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python 3.8, PyTorch 1.9) required to replicate the experiments. |
| Experiment Setup | Yes | The experimental setup is as follows: the rewards for all arms in all experiments are generated from the heavy tailed Student’s-t distribution. The shifted mean of each arm is given using the following rule: µ(i) = 2 − (i − 1/K)0.6 for i = 2, ..., K and µ(1) = 2. The number of degrees of freedom for the first two figures from the left is ν = 3 which corresponds to ε = 1 and υε = 3. The third figure uses ν = 2 which has infinite variance and for ε = 0.85, an upper bound is υε = 50. Smaller values for τ and larger values for h will have sharper bounds, and can be chosen depending on the tolerance to the cost of initial exploration. One choice parameters used is τ = 0.05 and h = 0.7. Further, γ(> 1) in Algorithm 3 is set to 1.1; this can be tuned further to improve performance. |