Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Online Learning with Abstention
Authors: Corinna Cortes, Giulia DeSalvo, Claudio Gentile, Mehryar Mohri, Scott Yang
ICML 2018 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We further report the results of a series of experiments demonstrating that UCB-GT largely outperforms that extension of UCB-N, as well as other standard baselines. |
| Researcher Affiliation | Collaboration | 1Google Research, New York, NY. 2INRIA Lille Nord Europe. 3Courant Institute of Mathematical Sciences, New York, NY. 4D. E. Shaw & Co., New York, NY. |
| Pseudocode | Yes | ALGORITHM 1: UCB-NT... ALGORITHM 2: UCB-GT |
| Open Source Code | No | The paper does not provide any links to open-source code for the methodology described. |
| Open Datasets | Yes | We used the following eight datasets from the UCI data repository: HIGGS, phishing, ijcnn, covtype, eye, skin, cod-rna, and guide. We also used the CIFAR dataset from (Krizhevsky et al., 2009) |
| Dataset Splits | No | The paper does not explicitly provide training/validation/test dataset splits. It mentions 'averaged over five random draws of the data'. |
| Hardware Specification | No | The paper does not provide specific hardware details used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | The predictors h are hyperplanes centered at the origin whose normal vector in Rd is drawn randomly from the Gaussian distribution, N(0, 1)d, where d is the dimension of the feature space of the dataset. The abstention functions r are concentric annuli around the origin with radii in (0, d 20 . . . , d). For each dataset, we generated a total of K = 2,100 experts and all the algorithms were tested for a total of T = 10,000 rounds. We report these results for c 2 {0.1, 0.2, 0.3}. |