Making Sense of Dependence: Efficient Black-box Explanations Using Dependence Measure
Authors: Paul Novello, Thomas FEL, David Vigouroux
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that HSIC is up to 8 times faster than the previous best black-box attribution methods while being as faithful. Indeed, we improve or match the state-of-the-art of both black-box and white-box attribution methods for several fidelity metrics on Imagenet with various recent model architectures. |
| Researcher Affiliation | Collaboration | Paul Novello 1 2 Thomas Fel 2 3 David Vigouroux 1 2 1 IRT Saint Exupery, France, 2 Artificial and Natural Intelligence Toulouse Institute, Université de Toulouse, France 3 Carney Institute for Brain Science, Brown University, USA paul.novello@irt-saintexupery.com |
| Pseudocode | Yes | Algorithm 1 Explanations using HSIC-based sensitivity analysis as attribution method |
| Open Source Code | Yes | Our implementation is available at https://github.com/paulnovello/HSIC-Attribution-Method. |
| Open Datasets | Yes | In Section 4.1, we compute explanations of the predictions in the ILSVRC-2012 [10] classification task (Image Net)... In Section 4.3, we evaluate HSIC attribution method to explain object detection on COCO dataset [28] with YOl Ov4 [35]. |
| Dataset Splits | Yes | In Table 1, we report the results of several different attribution methods for explaining the classification of Mobile Net [40], Res Net50 [22], Efficient Net [53] and VGG16 [44] on 1000 images sampled from the Image Net validation dataset. ... The explanations for 1, 000 validation images are evaluated with the Deletion, Insertion, and µFidelity metrics. |
| Hardware Specification | Yes | The execution times are averaged over 100 explanations of Res Net50 predictions with an RTX Quadro 8000 GPU. ... Execution times are averaged on the 1, 000 images on RTX 3080 GPUs. |
| Software Dependencies | No | The paper mentions using 'tensorflow [1] with the keras API [6]' but does not provide specific version numbers for these or any other software components used in the experiments. |
| Experiment Setup | Yes | We introduce two variants of our method, Hp i eff. and Hp i acc. The words "eff" and "acc" stand for efficient and accurate because we use p = 764 and p = 1536 samples, respectively. We use our method with a grid size of 7 × 7 (d = 49). |