Calibration of Neural Networks using Splines
Authors: Kartik Gupta, Amir Rahimi, Thalaiyasingam Ajanthan, Thomas Mensink, Cristian Sminchisescu, Richard Hartley
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We tested our method against existing calibration approaches on various image classification datasets and our spline-based recalibration approach consistently outperforms existing methods on KS error as well as other commonly used calibration measures. |
| Researcher Affiliation | Collaboration | 1Australian National University, 2Data61, CSIRO, 3Google Research |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Open-source implementation available at https://github.com/kartikgupta-at-anu/ spline-calibration |
| Open Datasets | Yes | We evaluate our proposed calibration method on four different imageclassification datasets namely CIFAR-10/100 (Krizhevsky et al. (2009)), SVHN (Netzer et al. (2011)) and Image Net (Deng et al. (2009)). |
| Dataset Splits | Yes | We use the pretrained network logits for spline fitting where we choose validation set as the calibration set, similar to the standard practice. Our final results for calibration are then reported on the test set of all datasets. Since Image Net does not comprise the validation set, test set is divided into two halves: calibration set and test set. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware (e.g., GPU models, CPU types) used to run its experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies such as libraries or frameworks. |
| Experiment Setup | Yes | We use the natural cubic spline fitting method (that is, cubic splines with linear run-out) with 6 knots for all our experiments. Further experimental details are provided in the supplementary. Note, for this experiment we use 14 knots for spline fitting. Note, for this experiment we use 13 knots for spline fitting. Note, for this experiment we use 6 knots for spline fitting. |