Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Generalized Fast Exact Conformalization
Authors: Diyang Li
NeurIPS 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide experimental results on real-world benchmarks to validate our derived algorithm.The significant speedups of our algorithm as compared to the existing standard methods are demonstrated across numerous benchmarks. |
| Researcher Affiliation | Academia | Diyang Li Cornell University EMAIL |
| Pseudocode | Yes | Algorithm 1 Fast Exact Conformalization Algorithm |
| Open Source Code | No | This paper does not release new assets. |
| Open Datasets | Yes | Our experiments were conducted using real-world datasets. We employ real-world datasets from Open ML [29] and UCI repository [30] in simulations. |
| Dataset Splits | Yes | We randomly partition the dataset into training set, testing set, and calibration set (used in SCP) with 70%, 10%, and 20% of the total samples. |
| Hardware Specification | Yes | All experiments presented in this study were conducted on a workstation running the Ubuntu 18.04 operating system, equipped with Intel Xeon Gold 5218R CPU 64 and 60.9 GB of RAM. |
| Software Dependencies | No | We integrate a system of ordinary differential equations using lsoda from the FORTRAN library, where an interface for Sci Py is available using the odepack. |
| Experiment Setup | Yes | The concrete parameter settings of ODE solver are shown in the Table 3, wherein the numerical solver exploit the Runge-Kutta method of order 4 or 5.We use the conformity score function Ai = |yi ηw (xi)|. |