Selective Classification for Deep Neural Networks
Authors: Yonatan Geifman, Ran El-Yaniv
NeurIPS 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results over CIFAR and Image Net convincingly demonstrate the viability of our method, which opens up possibilities to operate DNNs in mission-critical applications. For example, using our method an unprecedented 2% error in top-5 Image Net classification can be guaranteed with probability 99.9%, and almost 60% test coverage. |
| Researcher Affiliation | Academia | Yonatan Geifman Computer Science Department Technion Israel Institute of Technology yonatan.g@cs.technion.ac.il Ran El-Yaniv Computer Science Department Technion Israel Institute of Technology rani@cs.technion.ac.il |
| Pseudocode | Yes | Algorithm 1 Selection with Guaranteed Risk (SGR) |
| Open Source Code | No | The paper does not provide any explicit statements about open-sourcing code or links to a code repository. |
| Open Datasets | Yes | Using the well-known VGG-16 architecture, we apply our method on CIFAR-10, CIFAR-100 and Image Net (on Image Net we also apply the RESNET-50 architecture). [...] We now consider CIFAR-10; see [14] for details. [...] We used an already trained Image-Net VGG-16 model based on ILSVRC2014 [16]. |
| Dataset Splits | Yes | We applied the SGR algorithm on f10 with the SR confidence-rating function, where the training set for SGR, Sm, was taken as half of the standard CIFAR-10 validation set that was randomly split to two equal parts. The other half, which was not consumed by SGR for training, was reserved for testing the resulting bounds. Thus, this training and test sets where each of approximately 5000 samples. [...] We repeated the same experimental design but now the sizes of the training and test set were approximately 25,000. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments (e.g., GPU models, CPU types, or memory). |
| Software Dependencies | No | The paper mentions using VGG and RESNET architectures but does not specify any software dependencies with version numbers (e.g., specific deep learning frameworks like TensorFlow or PyTorch with their versions). |
| Experiment Setup | Yes | We used data augmentation containing horizontal flips, vertical and horizontal shifts, and rotations, and trained using SGD with momentum of 0.9, initial learning rate of 0.1, and weight decay of 0.0005. We multiplicatively dropped the learning rate by 0.5 every 25 epochs, and trained for 250 epochs. |