Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
Authors: Kimin Lee, Honglak Lee, Kibok Lee, Jinwoo Shin
ICLR 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate its effectiveness using deep convolutional neural networks on various popular image datasets. We demonstrate the effectiveness of our proposed method using various datasets: CIFAR (Krizhevsky & Hinton, 2009), SVHN (Netzer et al., 2011), Image Net (Deng et al., 2009), LSUN (Yu et al., 2015) and synthetic (Gaussian) noise distribution. |
| Researcher Affiliation | Collaboration | Korea Advanced Institute of Science and Technology, Daejeon, Korea University of Michigan, Ann Arbor, MI 48109 Google Brain, Mountain View, CA 94043 |
| Pseudocode | Yes | Algorithm 1 Alternating minimization for detecting and generating out-of-distribution. |
| Open Source Code | Yes | Our code is available at https://github.com/alinlab/Confident_classifier. |
| Open Datasets | Yes | We demonstrate the effectiveness of our proposed method using various datasets: CIFAR (Krizhevsky & Hinton, 2009), SVHN (Netzer et al., 2011), Image Net (Deng et al., 2009), LSUN (Yu et al., 2015) and synthetic (Gaussian) noise distribution. |
| Dataset Splits | Yes | CIFAR-10 and SVHN datasets: the former consists of 50,000 training and 10,000 test images with 10 image classes, and the latter consists of 73,257 training and 26,032 test images with 10 digits. For each out-of-distribution dataset, we randomly select 1,000 images for tuning the penalty parameter β, mini-batch size and learning rate. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware (e.g., GPU models, CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper mentions the "Adam learning rule" and deep learning frameworks implicitly but does not provide specific version numbers for software dependencies (e.g., Python, PyTorch, TensorFlow, CUDA). |
| Experiment Setup | Yes | The penalty parameter is chosen from β {0, 0.1, . . . 1.9, 2}, the mini-batch size is chosen from {64, 128} and the learning rate is chosen from {0.001, 0.0005, 0.0002}. The optimal parameters are chosen to minimize the detection error on the validation set. We drop the learning rate by 0.1 at 60 epoch and models are trained for total 100 epochs. |