Distribution-free binary classification: prediction sets, confidence intervals and calibration
Authors: Chirag Gupta, Aleksandr Podkopaev, Aaditya Ramdas
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We study three notions of uncertainty quantification calibration, confidence intervals and prediction sets for binary classification in the distribution-free setting, that is without making any distributional assumptions on the data. With a focus towards calibration, we establish a tripod of theorems that connect these three notions for score-based classifiers. |
| Researcher Affiliation | Academia | Chirag Gupta 1, Aleksandr Podkopaev 1,2, Aaditya Ramdas1,2 Machine Learning Department1 Department of Statistics and Data Science2 Carnegie Mellon University {chiragg,podkopaev,aramdas}@cmu.edu |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. Methods are described through textual explanations and mathematical formulations. |
| Open Source Code | No | The paper does not provide concrete access to source code (e.g., specific repository link, explicit code release statement, or code in supplementary materials) for the methodology described. |
| Open Datasets | No | The paper is theoretical and does not conduct experiments with specific datasets, therefore it does not provide access information for a publicly available or open dataset. |
| Dataset Splits | No | The paper is theoretical and does not report on experiments with specific datasets, therefore it does not provide specific dataset split information needed to reproduce data partitioning. |
| Hardware Specification | No | The paper is theoretical and does not report on specific experiments, therefore it does not provide specific hardware details used for running experiments. |
| Software Dependencies | No | The paper is theoretical and does not report on specific experiments, therefore it does not provide specific ancillary software details (e.g., library or solver names with version numbers). |
| Experiment Setup | No | The paper is theoretical and does not report on specific experiments, therefore it does not contain specific experimental setup details (e.g., concrete hyperparameter values, training configurations, or system-level settings). |