Learning to Augment Distributions for Out-of-distribution Detection
Authors: Qizhou Wang, Zhen Fang, Yonggang Zhang, Feng Liu, Yixuan Li, Bo Han
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive evaluations across representative OOD detection setups, demonstrating the superiority of our DAL over its advanced counterparts. We conduct extensive experiments over representative OOD detection setups, revealing the open-world performance of our method toward effective OOD detection. For example, our DAL reduces the average FPR95 by 1.99 to 13.46 on CIFAR benchmarks compared with the conventional outlier exposure (Hendrycks et al., 2019). The empirical results comprehensively demonstrate our superiority over advanced counterparts, and the improvement is mainly attributed to our distributional-augmented learning framework. |
| Researcher Affiliation | Academia | Qizhou Wang1 Zhen Fang2 Yonggang Zhang1 Feng Liu3 Yixuan Li4 Bo Han1 1Department of Computer Science, Hong Kong Baptist University 2Australian Artificial Intelligence Institute, University of Technology Sydney 3School of Computing and Information Systems, The University of Melbourne 4Department of Computer Sciences, University of Wisconsin-Madison |
| Pseudocode | Yes | Algorithm 1 Distributional-Augmented OOD Learning (DAL) |
| Open Source Code | Yes | The code is publicly available at: https://github.com/tmlr-group/DAL. |
| Open Datasets | Yes | We mainly test DAL on the CIFAR (Krizhevsky and Hinton, 2009) benchmarks (as ID datasets). We adopt the 80 Million Tiny Images (Torralba et al., 2008) as the auxiliary OOD dataset; |
| Dataset Splits | No | The hyper-parameters are tuned based on the validation data, separated from the training ID and auxiliary OOD data, which is a common strategy in OOD detection with outlier exposure field (Hendrycks et al., 2019; Chen et al., 2021). |
| Hardware Specification | No | The paper does not specify the hardware used for running the experiments (e.g., specific GPU models, CPU types, or memory). |
| Software Dependencies | No | The paper mentions using Wide ResNet-40-2 and specific loss functions, but it does not specify version numbers for any software libraries, programming languages, or development environments (e.g., Python, PyTorch, TensorFlow, CUDA versions). |
| Experiment Setup | Yes | Pre-training Setups. We employ Wide Res Net-40-2 (Zagoruyko and Komodakis, 2016) trained for 200 epochs via empirical risk minimization, with a batch size 64, momentum 0.9, and initial learning rate 0.1. The learning rate is divided by 10 after 100 and 150 epochs. Hyper-parameters Setups. For CIFAR-10, DAL is run for 50 epochs with the ID batch size 128, the OOD batch size 256, the initial learning rate 0.07, γmax = 10, β = 0.01, ρ = 10, ps = 1, and α = 1. For CIFAR-100, DAL is run for 50 epochs with the ID batch size 128, the OOD batch size 256, the initial learning rate 0.07, γmax = 10, β = 0.005, ρ = 10, and ps = 1, and α = 1. For both cases, we employ cosine decay (Loshchilov and Hutter, 2017) for the model learning rate. |