Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
SkipPredict: When to Invest in Predictions for Scheduling
Authors: Rana Shahout, Michael Mitzenmacher
NeurIPS 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To gain more insight into when to invest in prediction and how Skip Predict compares to other policies, in this section we compare Skip Predict, SPRPT, 1bit and FCFS using simulation with realworld and synthetic traces in the setting of the single queue with Poisson arrivals. |
| Researcher Affiliation | Academia | Rana Shahout Harvard University Michael Mitzenmacher Harvard University |
| Pseudocode | No | The paper describes algorithms verbally and mathematically but does not include any explicitly labeled "Pseudocode" or "Algorithm" blocks with structured code-like steps. |
| Open Source Code | Yes | We provide our code and scripts to reproduce all the results in the paper and the Appendix with readme file contains instructions as supplemental material. |
| Open Datasets | Yes | For real-world traces, we used three traces from Amvrosiadis et al. (Amvrosiadis et al., 2018): Twosigma, Google, and Trinity. |
| Dataset Splits | No | The paper describes the overall simulation setup and how mean response times are calculated over many runs, but it does not specify explicit training, validation, and test dataset splits with percentages or sample counts in the typical machine learning sense. |
| Hardware Specification | Yes | The evaluation was performed on an AMD EPYC 7313 16-Core Processor running Ubuntu 20.04.6 LTS with Linux kernel 5.4.0 172-generic. |
| Software Dependencies | Yes | We implemented the simulation in Python 3.7.6. |
| Experiment Setup | Yes | The default costs for the external model are c1 = 0.5, c2 = 20, and in the server time are c1 = 0.05, c2 = 0.5. |