Do Feature Attribution Methods Correctly Attribute Features?
Authors: Yilun Zhou, Serena Booth, Marco Tulio Ribeiro, Julie Shah9623-9633
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using this procedure, we evaluate three common methods: saliency maps, attentions and rationales. We identify several deficiencies and add new perspectives to the growing body of evidence questioning the correctness and reliability of these methods applied on datasets in the wild. We further discuss possible avenues for remedy and recommend new attribution methods to be tested against ground truth before deployment. |
| Researcher Affiliation | Collaboration | Yilun Zhou1, Serena Booth1, Marco Tulio Ribeiro2, Julie Shah1 1MIT CSAIL 2Microsoft Research 1{yilun, serenabooth, julie a shah}@csail.mit.edu, 2marcotcr@microsoft.com |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code and appendix are available at https://yilunzhou.github.io/feature-attribution-evaluation/. |
| Open Datasets | Yes | Dataset: We curate our own dataset on bird species identification. First, we train a Res Net-34 model on CUB-200-2011 (Wah et al. 2011)... We modify the Beer Advocate dataset (Mc Auley, Leskovec, and Jurafsky 2012)... |
| Dataset Splits | Yes | Last, we split the 1,200 images per class into train/validation/test sets of 1000/100/100 images... further select 12,000 reviews split into train, validation, and test sets of sizes 10,000, 1,000 and 1,000 (shuffled differently for each experiment). |
| Hardware Specification | No | The paper does not specify any particular hardware (e.g., GPU model, CPU type, memory size) used for running the experiments. |
| Software Dependencies | No | The paper mentions software components and models like "Res Net-34 architecture", "Bi-LSTM network", "Gradient", "Smooth Grad", "Grad CAM", "LIME", "SHAP", "reinforcement learning (RL) model", and "continuous relaxation (CR) model" but does not provide specific version numbers for any libraries or frameworks (e.g., PyTorch, TensorFlow, scikit-learn versions). |
| Experiment Setup | Yes | The parameters are randomly initialized rather than pre-trained on Image Net... We reassign labels with r = 0.5... For the positive reviews, we change all the article words (a / an / the) to the , and for the negative reviews, we change these to a... train the models to produce rationales that match a target selection rate %Sel. For a mini-batch of B examples, we use λ PB i=1 len(rationalei)/ PB i=1 len(reviewi) Sel% , where λ > 0 is the regularization strength. We also removed the discontinuity penalty... |