Igeood: An Information Geometry Approach to Out-of-Distribution Detection
Authors: Eduardo Dadalto Camara Gomes, Florence Alberge, Pierre Duhamel, Pablo Piantanida
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we show that IGEOOD outperforms competing state-of-the-art methods on a variety of network architectures and datasets. (Abstract) / 4 EXPERIMENTAL RESULTS We show the effectiveness of IGEOOD comparing to state-of-the-art methods. Details about the experimental setup and additional results are given in appendices (see Sections C, D, and E). |
| Researcher Affiliation | Academia | Eduardo D. C. Gomes, Florence Alberge & Pierre Duhamel Laboratoire des signaux et syst emes (L2S) Universit e Paris-Saclay CNRS Centrale Sup elec 91190, Gif-sur-Yvette, France. / Pablo Piantanida International Laboratory on Learning Systems (ILLS) Mc Gill ETS MILA CNRS Universit e Paris-Saclay Centrale Sup elec H3C 1K3 Quebec, Canada |
| Pseudocode | Yes | B IGEOOD ALGORITHMS AND COMPUTATION DETAILS / Algorithm 1: Evaluating IGEOOD score based on the logits. / Algorithm 2: Evaluating feature-wise IGEOOD score. / Algorithm 3: Evaluating feature-wise IGEOOD+ score. |
| Open Source Code | Yes | Our code is publicly available at https://github.com/edadaltocg/Igeood. |
| Open Datasets | Yes | We take as in-distribution data images from CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 and SVHN (Netzer et al., 2011) datasets. (Section 4) / C.3 DATASETS: Provides detailed descriptions and original citations for CIFAR-10, CIFAR-100, SVHN, Tiny-Image Net, LSUN, i SUN, Textures (DTD), Chars74K, Places365, Gaussian. |
| Dataset Splits | Yes | For the BLACK-BOX and GREY-BOX experimental settings, we tune hyperparameters for all of the OOD detectors only based on the DNN classifier architecture, the in-distribution dataset, and a validation dataset. The i SUN (Xu et al., 2015) dataset is chosen as a source of OOD validation data, independently from OOD test data. (Section 4.1) |
| Hardware Specification | No | Not found. The paper mentions 'one GPU' and 'one high-end GPU' but does not specify the exact model (e.g., NVIDIA A100, V100, RTX 3090) or other hardware components like CPU or memory for reproducibility. |
| Software Dependencies | No | Not found. While the paper mentions using SGD, cross-entropy loss, and implicitly deep learning frameworks, it does not specify any software names with version numbers (e.g., Python 3.x, PyTorch 1.x, CUDA 11.x). |
| Experiment Setup | Yes | We train each model by minimizing the cross-entropy loss using SGD with Nesterov momentum equal to 0.9, weight decay equal to 0.0001, and a multi-step learning rate schedule starting at 0.1 for 300 epochs. (C.1) / We used a constant learning rate of 0.01 and a batch size of 128 for 100 epochs. (B.1) / For temperature T, we ran a Bayesian optimization for 500 epochs in the interval of temperature values between 1 and 1000... For the input pre-processing noise magnitude ε tuning, we ran a grid search optimization with 21 equally spaced values in the interval [0, 0.002]. (E.2) / Table 7: Best temperatures T for the BLACK-BOX setup, best temperature and noise magnitude (T, ε) for the GREY-BOX setup, and best ε for the Mahalanobis score and (T, ε) for IGEOOD and IGEOOD+ in the WHITE-BOX setup with adversarial tuning. |