Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Adapting Contrastive Language-Image Pretrained (CLIP) Models for Out-of-Distribution Detection
Authors: Nikolas Adaloglou, Felix Michels, Tim Kaiser, Markus Kollmann
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present a comprehensive experimental study on pre-trained feature extractors for visual out-of-distribution (OOD) detection, focusing on leveraging contrastive language-image pretrained (CLIP) models. |
| Researcher Affiliation | Academia | Nikolas Adaloglou EMAIL Heinrich Heine University, Duesseldorf Felix Michels EMAIL Heinrich Heine University, Duesseldorf Tim Kaiser EMAIL Heinrich Heine University, Duesseldorf Markus Kollman EMAIL Heinrich Heine University, Duesseldorf |
| Pseudocode | No | The paper describes methods using text and mathematical equations, such as equations (1) to (8), but does not contain a structured pseudocode or algorithm block. |
| Open Source Code | Yes | The code is available at https://github.com/HHU-MMBS/plp-official-tmlr2024. |
| Open Datasets | Yes | Table 2: An overview of the number of classes and the number of samples on the considered datasets. ... In-distribution datasets CIFAR10 10 50K 10K CIFAR100 100 50K 10K Image Net 1K 1.28M 50K Out-distribution datasets i Naturalist 110 10K SUN 50 10K Places 50 10K IN-O 200 2K NINCO 64 5.88K Texture 47 5.54K CIFAR10-A 10 1000 CIFAR10-AS 10 1000 |
| Dataset Splits | Yes | Table 2: An overview of the number of classes and the number of samples on the considered datasets. ... In-distribution datasets CIFAR10 10 50K 10K CIFAR100 100 50K 10K Image Net 1K 1.28M 50K Out-distribution datasets i Naturalist 110 10K SUN 50 10K Places 50 10K IN-O 200 2K NINCO 64 5.88K Texture 47 5.54K CIFAR10-A 10 1000 CIFAR10-AS 10 1000 |
| Hardware Specification | Yes | All the experiments were carried out in a single NVIDIA A100 with 40GB VRAM. |
| Software Dependencies | No | The paper mentions using the Adam optimizer and describes training parameters (e.g., learning rate, batch size, weight decay), but does not provide specific version numbers for software dependencies like programming languages or libraries (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | Yes | We used the Adam optimizer (Kingma & Ba, 2014) with a mini-batch size of 256 for CIFAR10 and CIFAR100 and 8192 for Image Net and trained for 100 epochs with a weight decay of 10-3. The learning rate is set to 10-3 (mini-batch size)/256 with a linear warm-up over the first ten epochs and cosine decay after that. |