Locally Invariant Explanations: Towards Stable and Unidirectional Explanations through Local Invariant Learning
Authors: Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Kartik Ahuja, Vijay Arya
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we show on tabular, image and text data that the quality of our explanations with neighborhoods formed using random perturbations are much better than LIME and in some cases even comparable to other methods that use realistic neighbors sampled from the data manifold. |
| Researcher Affiliation | Collaboration | Amit Dhurandhar IBM Research Yorktown Heights, USA adhuran@us.ibm.com Karthikeyan Natesan Ramamurthy IBM Research Yorktown Heights, USA knatesa@us.ibm.com Kartik Ahuja Mila Montreal, Canada kartik.ahuja@mila.quebec Vijay Arya IBM Research Bangalore, India vijay.arya@in.ibm.com |
| Pseudocode | Yes | Algorithm 1: Locally Invariant EXplanations (LINEX). |
| Open Source Code | No | The paper does not contain an explicit statement or link indicating that the source code for the described methodology (LINEX) is publicly available. |
| Open Datasets | Yes | We test our method on five real world datasets covering all three modalities: IRIS (Tabular) [12], Medical Expenditure Panel Survey (Tabular) [1], Fashion MNIST (Image) [50], CIFAR10 (Image) [31] and Rotten Tomatoes reviews (Text) [38]. |
| Dataset Splits | No | In other cases except FMNIST and CIFAR10 which come with their own test partition we randomly split the datasets into 80/20% train/test partition and average results for the local explanations over this test partition. |
| Hardware Specification | Yes | The results were generated on Linux machines with 56 cores and 242 GB RAM. ... Realistic neighborhood generation can be time consuming especially for Me LIME since generators have to be trained which may take up to an hour using a single GPU for datasets such as FMNIST. |
| Software Dependencies | No | The paper mentions software components like 'Count Vectorizer and Tfidf Transformer and a sklearn Naive Bayes classifier' and refers to 'KDEGen' and 'VAEGen' (Section D.1), but it does not specify version numbers for these software packages or libraries. |
| Experiment Setup | Yes | We set perturbation neighborhood sizes 10 (IRIS), 500 (MEPS), 100 (FMNIST-random), 500 (FMNIST-realistic), 100 (CIFAR10-random), 500 (CIFAR10-realistic), 100 (Rotten tomatoes) for generating local explanations. We also use 3, 10, 10, 10, 5 as exemplar neighborhood sizes to compute GI, CI and Υ metrics for the five datasets respectively. We also use 5 sparse explanations for all cases except FMNIST and CIFAR10 with realistic perturbations where we follow Me LIME and generate a dense explanation using ridge penalty with penalty multiplier value of 0.001. The ℓ bound γ in Algorithm 1 is set as the maximum absolute value of linear coefficient computed by running LIME/Me LIME in the two individual environments. |