A General Framework For Detecting Anomalous Inputs to DNN Classifiers
Authors: Jayaram Raghuram, Varun Chandrasekaran, Somesh Jha, Suman Banerjee
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the proposed methods on well-known image classification datasets with strong adversarial attacks and OOD inputs, including an adaptive attack that uses the internal layer representations of the DNN (often not considered in prior work). Comparisons with five recently-proposed competing detection methods demonstrates the effectiveness of our method in detecting adversarial and OOD inputs. and 6. Experimental Results We evaluated JTLA on the following well-known image classification datasets: CIFAR-10 (Krizhevsky et al., 2009), SVHN (Netzer et al., 2011), and MNIST (Le Cun et al., 1998). |
| Researcher Affiliation | Collaboration | 1Computer Sciences, University of Wisconsin, Madison, USA. 2Xai Pient Inc., Princeton, NJ, USA. |
| Pseudocode | Yes | Algorithm 1 Meta-algorithm for Anomaly Detection |
| Open Source Code | Yes | The code base associated with our work can be found at: https://github.com/jayaram-r/ adversarial-detection. |
| Open Datasets | Yes | We evaluated JTLA on the following well-known image classification datasets: CIFAR-10 (Krizhevsky et al., 2009), SVHN (Netzer et al., 2011), and MNIST (Le Cun et al., 1998). |
| Dataset Splits | Yes | We performed class-stratified 5-folds cross-validation on the test partition provided by the datasets; the training folds are used for estimating the detector parameters, and the test folds are used solely for calculating performance metrics (which are then averaged across the test folds). |
| Hardware Specification | No | The paper does not specify the hardware used for experiments (e.g., specific GPU/CPU models, memory). |
| Software Dependencies | No | The paper mentions software like the 'Foolbox library' and 'NNDescent' but does not provide specific version numbers for them or other dependencies like Python or deep learning frameworks. |
| Experiment Setup | Yes | We used the training partition provided by the datasets for training standard CNN architectures, including a Resnet for CIFAR-10. We performed class-stratified 5-folds cross-validation on the test partition provided by the datasets; the training folds are used for estimating the detector parameters, and the test folds are used solely for calculating performance metrics (which are then averaged across the test folds). ... The number of nearest neighbors is the only hyperparameter of the proposed instantiation of JTLA. This is set to be a function of the number of in-distribution training samples n using the heuristic k = n0.4 . |