Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Online metric algorithms with untrusted predictions
Authors: Antonios Antoniadis, Christian Coester, Marek Elias, Adam Polak, Bertrand Simon
ICML 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we present an empirical evaluation of our methods on real world datasets, which suggests practicality. (Abstract); 5. Experiments We evaluate the practicality of our approach on real-world datasets for two MTS: caching and ice cream problem. (Section 5) |
| Researcher Affiliation | Academia | 1Saarland University and Max-Planck Institute for Informatics, Saarbr ucken, Germany 2CWI, Amsterdam, Netherlands 3EPFL, Lausanne, Switzerland 4Faculty of Mathematics and Computer Science, Jagiellonian University, Krak ow, Poland 5University of Bremen, Bremen, Germany. |
| Pseudocode | Yes | We provide a pseudocode in the supplementary material. (Footnote 2, Section 4); Algorithm 1: MIN det (Fiat et al., 1994) and Algorithm 2: MIN rand (Blum & Burch, 2000) are present in Section 2.1. |
| Open Source Code | Yes | The source code and datasets are available at Git Hub5. https://github.com/adampolak/ mts-with-predictions (Section 5) |
| Open Datasets | Yes | The source code and datasets are available at Git Hub5. https://github.com/adampolak/ mts-with-predictions (Section 5); BK dataset comes from a former social network Bright Kite (Cho et al., 2011). (Section 5.1); Citi dataset comes from a bike sharing platform Citi Bike. (Section 5.1) |
| Dataset Splits | No | The paper describes the datasets used (BK, Citi) but does not provide specific details on training, validation, and test splits (e.g., percentages, sample counts, or explicit splitting methodology). |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., CPU, GPU models, memory, or cloud instances) used for running the experiments. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers (e.g., programming languages, libraries, or specific solvers with their versions) that would be needed for replication. |
| Experiment Setup | Yes | We set the parameters to γ = 1 + 0.01 and ϵ = 0.01. These values, chosen from {10 i : i = 0, . . . , 4}, happen to be consistently the best choice in all our experimental settings. (Section 5.1, 'Algorithms' paragraph) |