Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives
Authors: Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, Payel Das
NeurIPS 2018 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper we propose a novel method that provides contrastive explanations justifying the classification of an input by a black box classifier such as a deep neural network. ... We validate our approach on three real datasets obtained from diverse domains; namely, a handwritten digits dataset MNIST, a large procurement fraud dataset and a brain activity strength dataset. In all three cases, we witness the power of our approach in generating precise explanations that are also easy for human experts to understand and evaluate. |
| Researcher Affiliation | Collaboration | Amit Dhurandhar IBM Research Yorktown Heights, NY 10598 EMAIL; Pin-Yu Chen IBM Research Yorktown Heights, NY 10598 EMAIL; Ronny Luss IBM Research Yorktown Heights, NY 10598 EMAIL; Chun-Chen Tu University of Michigan Ann Arbor, MI 48109 EMAIL; Paishun Ting University of Michigan Ann Arbor, MI 48109 EMAIL; Karthikeyan Shanmugam IBM Research Yorktown Heights, NY 10598 EMAIL; Payel Das IBM Research Yorktown Heights, NY 10598 EMAIL |
| Pseudocode | Yes | Algorithm 1 Contrastive Explanations Method (CEM) |
| Open Source Code | Yes | Code at https://github.com/IBM/Contrastive-Explanation-Method |
| Open Datasets | Yes | a handwritten digits dataset MNIST [40]... a brain functional MRI (f MRI) imaging dataset from the publicly accessible Autism Brain Imaging Data Exchange (ABIDE) I database [11] |
| Dataset Splits | Yes | The handwritten digits are classified using a feed-forward convolutional neural network (CNN) trained on 60,000 training images from the MNIST benchmark dataset. ... The 10-fold cross validation accuracy of the network was high (91.6%). ... The leave-one-out cross validation testing accuracy is around 61.17%. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory amounts used for running experiments. |
| Software Dependencies | No | The paper mentions 'Tensor Flow' but does not specify a version number. It also refers to 'LRP experiments used the toolbox from [21]' and 'LIME code was adapted from https://github.com/marcotcr/lime' without providing specific software versions for these tools. |
| Experiment Setup | Yes | The parameter κ 0 is a confidence parameter that controls the separation between [Pred(x0 + δ)]t0 and maxi =t0[Pred(x0 + δ)]i. The parameters c, β, γ, 0 are the associated regularization coefficients. We apply a projected fast iterative shrinkage-thresholding algorithm (FISTA) [2] to solve problems (1) and (3). The CNN has two sets of convolution-convolution-pooling layers, followed by three fully-connected layers. We trained a three-layer neural network with fully connected layers, 512 rectified linear units and a three-way softmax function. The parameters of the model were regularized by an elastic-net regularizer. |