When Explanations Lie: Why Many Modified BP Attributions Fail
Authors: Leon Sixt, Maximilian Granz, Tim Landgraf
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We find empirically that the explanations of all mentioned methods, except for Deep LIFT, are independent of the parameters of later layers. We provide theoretical insights for this surprising behavior and also analyze why Deep LIFT does not suffer from this limitation. Empirically, we measure how information of later layers is ignored by using our new metric, cosine similarity convergence (CSC). The paper provides a framework to assess the faithfulness of new and existing modiļ¬ed BP methods theoretically and empirically. |
| Researcher Affiliation | Academia | 1Dahlem Center of Machine Learning and Robotics, Freie Universit at Berlin, Germany. Correspondence to: Leon Sixt <leon.sixt@fu-berlin.de>. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | For code see: github.com/berleon/when-explanations-lie |
| Open Datasets | Yes | We report results on a small network trained on CIFAR-10 (4x conv., 2x dense, see appendix D), a VGG-16 (Simonyan & Zisserman, 2014), and Res Net-50 (He et al., 2016). The last two are trained on the Image Net dataset (Russakovsky et al., 2015), the standard dataset to evaluate attribution methods. |
| Dataset Splits | No | The paper mentions 'All results were computed on 200 images from the validation set.' but does not specify the training/test/validation dataset splits (e.g., percentages or sample counts for each split). |
| Hardware Specification | Yes | We are also grateful to Nvidia for a Titan Xp and to ZEDAT for access their HPC system. |
| Software Dependencies | No | We used the implementation from the innvestigate and deeplift package (Alber et al., 2019; Shrikumar et al., 2017) and added support for residual connections. However, specific version numbers for these packages or other software dependencies are not provided. |
| Experiment Setup | No | The paper does not explicitly provide details about the experimental setup such as hyperparameters (e.g., learning rate, batch size, number of epochs) or system-level training settings for the models used. |