Input Complexity and Out-of-distribution Detection with Likelihood-based Generative Models
Authors: Joan Serrà, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F. Núñez, Jordi Luque
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We report a set of experiments supporting this hypothesis, and use an estimate of input complexity to derive an efficient and parameter-free OOD score... We base our experiments on an extensive collection of alternatives, including a pool of 12 data sets, two conceptually-different generative models, increasing model sizes, and three variants of complexity estimates. |
| Researcher Affiliation | Collaboration | 1 Dolby Laboratories, Barcelona, Spain 2 Telef onica Research, Barcelona, Spain 3 Universitat Polit ecnica de Catalunya, Barcelona, Spain 4 Universitat Pompeu Fabra, Barcelona, Spain |
| Pseudocode | No | The paper describes the methodology in narrative text and mathematical formulas but does not provide structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper mentions using and implementing based on existing open-source projects (e.g., Glow's original implementation, PyTorch), but it does not provide a link or explicit statement about releasing the source code for the methodology presented in this paper. |
| Open Datasets | Yes | In our experiments, we employ well-known, publicly-available data sets. ... The complete list of data sets is available in Table 3. ... CIFAR10 (Krizhevsky, 2009) ... MNIST (Le Cun et al., 2010) ... SVHN (Netzer et al., 2011) |
| Dataset Splits | Yes | In order to split the data between train, validation, and test, we follow two simple rules: (1) if the data set contains some predefined train and test splits, we respect them and create a validation split using a random 10% of the training data; (2) if no predefined splits are available, we create them by randomly assigning 80% of the data to the train split and 10% to both validation and test splits. |
| Hardware Specification | Yes | All models have been trained with a single NVIDIA Ge Force GTX 1080Ti GPU. Training takes some hours under that setting. |
| Software Dependencies | Yes | We use Py Torch version 1.2.0 (Paszke et al., 2017). |
| Experiment Setup | Yes | We train both Glow and Pixel CNN++ using the Adam optimizer with an initial learning rate of 10 4. We reduce this initial value by a factor of 1/5 every time that the validation loss does not decrease during 5 consecutive epochs. The training finishes when the learning rate is reduced by factor of 1/100. The batch size of both models is set to 50. The final model weights are the ones yielding the best validation loss. |