STEP: Out-of-Distribution Detection in the Presence of Limited In-Distribution Labeled Data
Authors: Zhi Zhou, Lan-Zhe Guo, Zhanzhan Cheng, Yu-Feng Li, Shiliang Pu
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comprehensive experiments across various OOD detection benchmarks clearly show that our STEP approach outperforms other methods by a large margin and achieves remarkable detection performance on several benchmarks. ... We evaluate our approach with comprehensive experiments across various OOD detection benchmarks. Our proposal outperforms previous methods by a large margin and achieves remarkable detection performance on several benchmarks. |
| Researcher Affiliation | Collaboration | 1National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China {zhouz, guolz, liyf}@lamda.nju.edu.cn 2Hikvision Research Institute, Hangzhou, China {chengzhanzhan, pushiliang.hri}@hikvision.com |
| Pseudocode | Yes | Algorithm 1 Training Phase of STEP Input: Dl: ID labeled data set; Du: unlabeled mixed data set; K: number of neighbours Output: pre-trained backbone fθ( ); projctor P |
| Open Source Code | Yes | More details on implementation are provided in the supplementary material and our code has been open source 2. https://www.lamda.nju.edu.cn/code_step.ashx |
| Open Datasets | Yes | In-distribution Data Set. We use CIFAR-10 and CIFAR-100 [25] as ID data sets in our experiments. ... Out-of-distribution Data Set. We use Tiny Image Net data set [6] and Large-scale Scene Understanding data set [48] as OOD data sets. |
| Dataset Splits | Yes | For CIFAR-10, each image belongs to one of 10 classes, and we randomly sample 250 training images as ID labeled data Dl. ... For CIFAR-100, the size of image classes is 100, and we randomly sample 400 training images as ID labeled data. Dl. We add the remaining training images to the unlabeled data Du. ... We randomly draw several images from ID testing images and OOD images as the OOD validation set. The rest of the OOD images are added to unlabeled data Du and used as testing data. |
| Hardware Specification | Yes | All experiments are performed on one single NVIDIA 3090 graphics card. |
| Software Dependencies | No | The paper mentions using "Sim CLR [5]" and "Densenet-BC [21]" but does not specify software versions (e.g., Python, PyTorch, TensorFlow versions, or specific library versions). |
| Experiment Setup | Yes | Our backbone is trained by SOTA contrastive learning method Sim CLR [5] for 500 epochs. We set the learning rate to 10 3 with a cosine annealing strategy. For fair comparisons, each comparison method can use the pre-trained backbone model. ... The hyper-parameter K for STEP is set to 12 for all data set pairs. |