Online Learning with Sleeping Experts and Feedback Graphs
Authors: Corinna Cortes, Giulia Desalvo, Claudio Gentile, Mehryar Mohri, Scott Yang
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically validate our algorithm against multiple online abstention algorithms on several real-world datasets, showing substantial performance improvements. In Section 6, we corroborate our theoretical results for the abstention setting with extensive experiments against multiple online abstention algorithms on several real-world datasets, showing that substantial improvements are also achieved empirically. Figure 3 shows the averaged abstention loss L( )/t with standard deviations for the different abstention costs. In Appendix E, we show the plots of all the datasets we tested, where the same patterns recur. These experiments show that UCB-ABS outperforms UCB-NT and UCB on all datasets and it attains a better averaged loss than that of UCB-GT on most datasets. |
| Researcher Affiliation | Collaboration | 1Google Research, New York, NY; 2Courant Institute of Mathematical Sciences, New York, NY; 3D. E. Shaw & Co., New York, NY. |
| Pseudocode | Yes | ALGORITHM 1: AUER-N, ALGORITHM 2: The UCB-SLG algorithm., ALGORITHM 3: UCB-ABS. |
| Open Source Code | No | The paper does not contain any statement about releasing source code or provide a link to a code repository. |
| Open Datasets | Yes | We used the CIFAR dataset from Krizhevsky et al. (2009), where we extracted the first twenty-five principal components, and used eight UCI datasets: HIGGS, phishing, ijcnn, covtype, eye, skin, cod-rna, and guide. |
| Dataset Splits | No | The paper mentions averaging results over random draws of data and experts but does not provide specific train/validation/test dataset splits, percentages, or explicit methodologies for splitting data into these sets. |
| Hardware Specification | No | The paper does not specify any hardware details like GPU/CPU models, processor types, or memory used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies or version numbers for any libraries, frameworks, or programming languages used in the experiments. |
| Experiment Setup | No | The paper describes the setup for prediction and abstention functions (random hyperplanes, concentric annuli) and the range of abstention costs (c in {0.05, ..., 0.9}), but it does not provide specific hyperparameters like learning rates, batch sizes, optimizer settings, or number of epochs for model training. |