Exact Optimization of Conformal Predictors via Incremental and Decremental Learning
Authors: Giovanni Cherubin, Konstantinos Chatzikokolakis, Martin Jaggi
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our findings empirically, and discuss when methods are suitable for CP optimization. and We empirically compare our techniques with i) original implementations of full CP, and ii) the most computationally efficient CP modification, ICP. |
| Researcher Affiliation | Academia | 1Alan Turing Institute, London, UK 2University of Athens 3EPFL. |
| Pseudocode | Yes | Algorithm 1 CP: computing a p-value for (x, ˆy) |
| Open Source Code | Yes | Code to reproduce the experiments: https://github. com/gchers/exact-cp-optimization. |
| Open Datasets | Yes | (In Appendix G, we further compare CP and ICP on the MNIST dataset.) |
| Dataset Splits | No | The paper uses generated data for its main experiments and does not specify explicit train/validation/test splits, percentages, or cross-validation methodology for its evaluation. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments. |
| Software Dependencies | No | The paper mentions using the 'scikit-learn library' but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | For every training size n, chosen in the space [10, 105], we train the CP with a nonconformity measure, and use it to predict 100 test points. We set a timeout of 10 hours... We generate data for a binary classification problem with 30 features, by using the make classification() routine of the scikit-learn library. and We fix t/n = 0.5. |