Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancement
Authors: Kai Xu, Rongyu Chen, Gianni Franchi, Angela Yao
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experiment with CIFAR10, CIFAR100 (Krizhevsky, 2009), and Image Net-1K (Deng et al., 2009) ID data sources. [...] Metrics. We evaluate with (1) FPR@95, which measures the false positive rate at a fixed true positive rate of 95%, where lower scores are better and (2) AUROC (Area under the ROC curve). |
| Researcher Affiliation | Academia | Kai Xu1, Rongyu Chen1, Gianni Franchi2, Angela Yao1 1National University of Singapore 2U2IS, ENSTA Paris, Institut polytechnique de Paris |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | Our code and models are available at https://github.com/kai422/SCALE. |
| Open Datasets | Yes | We experiment with CIFAR10, CIFAR100 (Krizhevsky, 2009), and Image Net-1K (Deng et al., 2009) ID data sources. [...] We used SVHN (Netzer et al., 2011), i SUN (Xu et al., 2015), Places365 (Zhou et al., 2018), and Textures (Cimpoi et al., 2014) as OOD datasets. |
| Dataset Splits | Yes | The Open OOD benchmark includes improved hyperparameter selection with a dedicated OOD validation set to prevent overfitting to the testing set." and "Table 2 uses p = 0.85 for SCALE and ASH-S, which is verified on the validation set. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for experiments, such as GPU or CPU models. |
| Software Dependencies | No | We adopted the Res Net-50 (He et al., 2016) model architecture and obtained the pretrained network from the torchvision library." (No version specified for torchvision or other libraries). |
| Experiment Setup | Yes | For training, we fine-tuned the torchvision pretrained model with ISH for 10 epochs with a cosine annealing learning rate schedule initiated at 0.003 and a minimum of 0. We additionally observed that using a smaller weight decay value (5e-6) enhances OOD detection performance. |