Where do Models go Wrong? Parameter-Space Saliency Maps for Explainability
Authors: Roman Levin, Manli Shu, Eitan Borgnia, Furong Huang, Micah Goldblum, Tom Goldstein
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we aim to validate the meaningfulness of our parameter saliency method. First, we verify that salient parameters are indeed responsible for misclassification by showing on the dataset level that turning them off improves predictions on the associated samples more than turning off the same number of random or least salient parameters. This experiment is similar in spirit to removing salient features to validate input-space saliency methods [5, 33]. |
| Researcher Affiliation | Academia | Roman Levin Department of Applied Mathematics University of Washington rilevin@uw.edu; Manli Shu Department of Computer Science University of Maryland manlis@cs.umd.edu; Eitan Borgnia Department of Computer Science University of Maryland eborgnia2@gmail.edu; Furong Huang Department of Computer Science University of Maryland furongh@cs.umd.edu; Micah Goldblum Center for Data Science New York University goldblum@nyu.edu; Tom Goldstein Department of Computer Science University of Maryland tomg@cs.umd.edu |
| Pseudocode | No | The paper describes its methods using mathematical formulas and descriptive text but does not include any explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code for computing parameter-saliency maps is available at https://github.com/ Levin Roman/parameter-space-saliency. |
| Open Datasets | Yes | We evaluate our saliency method in the context of image classification on CIFAR-10 [21] and Image Net [9]. |
| Dataset Splits | Yes | Images we use for visualization, unless otherwise specified, are sampled from Image Net validation set. Standardized filter-wise saliency profiles averaged over correctly classified samples in the Image Net validation set. |
| Hardware Specification | Yes | All experiments were run on NVIDIA V100 GPUs. |
| Software Dependencies | No | The paper mentions using PyTorch-related repositories for CIFAR-10 and ImageNet models, but does not provide specific version numbers for PyTorch, Python, or other relevant software dependencies. |
| Experiment Setup | Yes | For CIFAR-10 experiments, we used a Res Net-18 model trained for 200 epochs using SGD with a learning rate of 0.1, a momentum of 0.9, and a weight decay of 5e-4. For Image Net experiments, we used a Res Net-50 model trained for 100 epochs using SGD with a learning rate of 0.1, a momentum of 0.9, and a weight decay of 1e-4. |