Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Conformal Inference for Online Prediction with Arbitrary Distribution Shifts
Authors: Isaac Gibbs, Emmanuel J. Candès
JMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We test our techniques on two real-world datasets aimed at predicting stock market volatility and COVID-19 case counts and find that they are robust and adaptive to real-world distribution shifts. Keywords: Conformal inference, online prediction, distribution shift, prediction set, online convex optimization |
| Researcher Affiliation | Academia | Isaac Gibbs EMAIL Department of Statistics Stanford University Stanford, CA 94305, USA Emmanuel Cand es EMAIL Departments of Statistics and Mathematics Stanford University Stanford, CA 94305, USA |
| Pseudocode | Yes | Algorithm 1: Dt ACI, modified version of Algorithm 1 in Gradu et al. (2023). ... Algorithm 2: |
| Open Source Code | Yes | Code for reproducing these results can be found at https://github.com/isgibbs/Dt ACI. |
| Open Datasets | Yes | All data is obtained from a public repository made available by the DELPHI group at Carnegie Mellon (Reinhart et al. (2021)). |
| Dataset Splits | Yes | For our first real-world data example, we return to a stock market prediction task... At each time step, t, we use the most recent 1250 days of returns {Rs}t 1250 s<t to produce estimates... To compute prediction sets for county i, we define the conformity scores St,i := |d COt,i COt,i|/|COt 7,i COt,i| and counts nt := |{i : County i has available data at time t 1}|, and set ˆCt,i(αt,i) := c : |d COt,i c| |COt 7,i c| Quantile. |
| Hardware Specification | No | No specific hardware details (like GPU/CPU models, memory, or cloud instances) are mentioned in the paper. |
| Software Dependencies | No | The paper does not provide specific version numbers for software libraries, frameworks, or programming languages used in the experiments. |
| Experiment Setup | Yes | In all experiments the set of candidate γ values is taken to be {0.001, 0.002, 0.004, 0.008, 0.0160, 0.032, 0.064, 0.128}. ... In our experiments, we will set η and σ using the choice |I| = 500. ... The hyperparameters to be m = 40, η = q 4.2m , and r = 800000. ... We model the stock returns Rt := Pt Pt 1 Pt 1 as coming from a GARCH(1,1) design. ... using least-squares regression to fit the model COs,i βt 0 + j=1 λt j COs 7j,i + j=1 κt j Fs 7j,i, s = t 14 . . . , t, i = 1, . . . , 3243. |