Overcoming Brittleness in Pareto-Optimal Learning Augmented Algorithms
Authors: Alex Elenter, Spyros Angelopoulos, Christoph Dürr, Yanni LEFKI
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In Section 6 we give an experimental evaluation of all our algorithms, over both real data (Bitcoin exchange rates) and synthetic data, which validates the theoretical results and quantifies the obtained performance improvements. |
| Researcher Affiliation | Academia | Alex Elenter Sorbonne University, CNRS, LIP6 4 place Jussieu Paris, France 75005 alexelenter@gmail.com Spyros Angelopoulos International Laboratory on Learning Systems Montreal, Canada, and Sorbonne University, CNRS, LIP6 Paris, France 75005 spyros.angelopoulos@lip6.fr Christoph Dürr Sorbonne University, CNRS, LIP6 4 place Jussieu Paris, France 75005 christoph.durr@lip6.fr Yanni Lefki Institut Polytechnique de Paris Rte de Saclay Palaiseau, 91120, France yanni.lefki@gmail.com |
| Pseudocode | Yes | Algorithm 1 Algorithm PROFILE for FEASIBLE (F); also an online strategy if F is feasible (...) Algorithm 2 ADA-PO (adaptive Pareto-optimal) |
| Open Source Code | Yes | 5. Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: The code and data are provided in the supplemental material. We also provided a README file that explains how to execute and reproduce the code. |
| Open Datasets | Yes | 6 Experimental evaluation (...) We used exchange rates from Bitcoin (BTC) to USD; specifically, we used a list of the last 1000 daily exchange rates (finishing on May 20, 2024), defining as the prediction ˆp the maximum rate in the first 200 rates, and running the algorithm on a sequence consisting of the last 800 rates. (...) 5. Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We used Bitcoin exchange rates from Binance, which are publicly available, and fully explained how precisely it was obtained. |
| Dataset Splits | No | We used exchange rates from Bitcoin (BTC) to USD; specifically, we used a list of the last 1000 daily exchange rates (finishing on May 20, 2024), defining as the prediction ˆp the maximum rate in the first 200 rates, and running the algorithm on a sequence consisting of the last 800 rates. |
| Hardware Specification | No | F Computational setup The experiments are reproducible on any standard computer, and do not require any memory or computational power beyond the standard requirements. They run typically within few milliseconds. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions) required for reproduction. |
| Experiment Setup | Yes | 6 Experimental evaluation Profile setting. We use a profile F that consists of three intervals [q1 = 1, q2), [q2, q3) and [q3, q4 = M], where M = 100. The profile is defined in terms of the prediction ˆp, by choosing q2 = 0.9ˆp and q3 = 1.1ˆp. In addition, F is such that F([q1, q2)) = t1 = F([q3, q4]) = t3 = r, where r = 4 (larger than, but close to the optimal competitive ratio r ). Here, F([q2, q3)) = t2 < r is the smallest value such that F is feasible. To find t2, we use binary search in [1, r] in combination with PROFILE, and note that this depends on ˆp. (...) Recall that such sequence is of the form 1, . . . , p , 1, with infinitesimal increments up to p , simulated using a step equal to 0.01. |