WeiPer: OOD Detection using Weight Perturbations of Class Projections
Authors: Maximilian Granz, Manuel Heurich, Tim Landgraf
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate Wei Per on Open OOD using our proposed KL-divergence-based scoring function (KLD), MSP Hendrycks & Gimpel (2016), and Re Act Sun et al. (2021). Additionally, we conduct an ablation study to understand the influence of each component of Wei Per and analyze Wei Per s performance. Our results confirm that the weight perturbations allow Wei Per to outperform the competition on two out of eight benchmarks, demonstrating consistently better performance on near OOD tasks. |
| Researcher Affiliation | Academia | Maximilian Granz Institute for Computer Science Free University of Berlin Arnimallee 7 14195 Berlin maximilian.granz@fu-berlin.de Manuel Heurich Institute for Computer Science Free University of Berlin Arnimallee 7 14195 Berlin manuel.heurich@fu-berlin.de Tim Landgraf Institute for Computer Science Free University of Berlin Arnimallee 7 14195 Berlin tim.landgraf@fu-berlin.de |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks clearly labeled as 'Pseudocode' or 'Algorithm'. Figure 2 provides a visual explanation of the method but is not formatted as pseudocode. |
| Open Source Code | Yes | Our code is available at: https://github.com/mgranz/weiper. |
| Open Datasets | Yes | We evaluate Wei Per using the Open OOD Zhang et al. (2023b) framework that includes three vision benchmarks: CIFAR10 Krizhevsky (2009), CIFAR100 Krizhevsky (2009), and Image Net Deng et al. (2009). |
| Dataset Splits | Yes | The hyperparameters of our methods were tuned by finding the best combination over a predefined and discrete range of values on the Open OOD validation sets to assure a fair comparison to the competition (see Table 8). |
| Hardware Specification | Yes | All experiments are conducted on a local machine with the following key specifications: AMD EPYC 7543 (32-Core Processor) with 256GB RAM and 1x NVIDIA RTX A5000 (24GB VRAM). |
| Software Dependencies | No | The paper mentions that 'All required packages will be installed when setting up Open OOD' and discusses using standard preprocessing and the cross entropy objective. However, it does not explicitly list specific software dependencies with their version numbers (e.g., Python 3.x, PyTorch 1.x, CUDA x.x). |
| Experiment Setup | Yes | Setup. We evaluate Wei Per using the Open OOD Zhang et al. (2023b) framework that includes three vision benchmarks: CIFAR10 Krizhevsky (2009), CIFAR100 Krizhevsky (2009), and Image Net Deng et al. (2009). [...] The hyperparameters of our methods were tuned by finding the best combination over a predefined and discrete range of values on the Open OOD validation sets to assure a fair comparison to the competition (see Table 8). [...] The full list of hyperparameters is r and δ for the Wei Per application and nbins, λ1, λ2, s1, s2 for the KL divergence score function. |