Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers
Authors: Yonatan Geifman, Guy Uziel, Ran El-Yaniv
ICLR 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present extensive experiments indicating that the proposed algorithm provides uncertainty estimates that are consistently better than all known methods. We report on extensive experiments with four baseline methods (including all those mentioned above) and four image datasets. |
| Researcher Affiliation | Academia | Yonatan Geifman Computer Science Department Technion Israel Institute of Technology yonatan.g@cs.technion.ac.il Guy Uziel Computer Science Department Technion Israel Institute of Technology uzielguy@gmail.com Ran El-Yaniv Computer Science Department Technion Israel Institute of Technology rani@cs.technion.ac.il |
| Pseudocode | Yes | Algorithm 1 The Pointwise Early Stopping Algorithm for Confidence Scores (PES) |
| Open Source Code | No | The paper does not provide any explicit statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | CIFAR-10: The CIFAR-10 dataset (Krizhevsky & Hinton, 2009)... CIFAR-100: The CIFAR-100 dataset (Krizhevsky & Hinton, 2009)... Street View House Numbers (SVHN): The SVHN dataset (Netzer et al., 2011)... Image Net: The Image Net dataset (Deng et al., 2009)... |
| Dataset Splits | Yes | CIFAR-10: The CIFAR-10 dataset (Krizhevsky & Hinton, 2009) is an image classification dataset containing 50,000 training images and 10,000 test images that are classified into 10 categories. To generate an independent training set, which is required by PES, we randomly split the original validation set (in each dataset) into two parts and took a random 70% of the set for training our algorithm, using the remaining 30% for validation. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, memory, or cloud instance types used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependency details, such as programming languages, libraries, or frameworks with their version numbers, needed to replicate the experiment. |
| Experiment Setup | Yes | We trained the model for 250 epochs using SGD with a momentum value of 0.9. We used an initial learning rate of 0.1, a learning rate multiplicative drop of 0.5 every 20 epochs, and a batch size of 128. For Image Net dataset, we used the Resnet-18 architecture... we trained the model using SGD with a batch size of 256 and momentum of 0.9 for 90 epochs. We used a learning rate of 0.1, with a learning rate multiplicative decay of 0.1 every 30 epochs. |