Detecting Semantic Anomalies
Authors: Faruk Ahmed, Aaron Courville3154-3162
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We make progress on these benchmarks by exploring a multi-task learning based approach, showing that auxiliary objectives for improved semantic awareness result in improved semantic anomaly detection, with accompanying generalization benefits. In tables 3 and 4 we report average precision scores on CIFAR-10 and STL-10 for the baseline scoring methods MSP (Hendrycks and Gimpel, 2017) and ODIN (Liang, Li, and Srikant, 2018). |
| Researcher Affiliation | Academia | Faruk Ahmed,1 Aaron Courville1,2 1Mila, Universit e de Montr eal 2CIFAR Fellow {faruk.ahmed, aaron.courville}@umontreal.ca |
| Pseudocode | No | The paper includes mathematical formulas but does not provide any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not include any explicit statement about making the source code available or provide a link to a code repository. |
| Open Datasets | Yes | We shall explore this setting with CIFAR-10 and STL-10, and recommend this as the default benchmark for evaluating anomaly detection in the context of object recognition. We propose a suite of tasks comprised of subsets of ILSVRC2012 (Russakovsky et al., 2015), with fine-grained subcategories. |
| Dataset Splits | Yes | λ is a hyper-parameter we learn by optimizing for classification accuracy on the validation set. For tuning λ, we treat 20% of the data in the training sets as validation, and the remaining 80% for training. |
| Hardware Specification | Yes | We train in parallel on 4 Pascal V100 GPUs with batches of size 128 on each. |
| Software Dependencies | No | The paper mentions using a Wide Res Net architecture and common optimizers like stochastic gradient descent with Nesterov’s momentum, but it does not specify software versions for libraries (e.g., PyTorch, TensorFlow) or programming languages (e.g., Python). |
| Experiment Setup | Yes | Our base network for all CIFAR-10 experiments is a Wide Res Net (Zagoruyko and Komodakis, 2016) with 28 convolutional layers and a widening factor of 10 (WRN-28-10) with the recommended dropout rate of 0.3. Following Zagoruyko and Komodakis (2016), we train for 200 epochs, with an initial learning rate of 0.1 which is scaled down by 5 at the 60th, 120th, and 160th epochs, using stochastic gradient descent with Nesterov s momentum at 0.9. We train in parallel on 4 Pascal V100 GPUs with batches of size 128 on each. For ODIN, it is unclear how to choose the hyperparameters for temperature scaling and the weight for adversarial perturbation without assuming access to anomalous examples, an assumption we consider unrealistic in most practical settings. We fix T = 1000, ϵ = 5e-5 for all experiments, following the most common setting. |