A Unified Approach to Interpreting Model Predictions
Authors: Scott M. Lundberg, Su-In Lee
NeurIPS 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluated the benefits of SHAP values using the Kernel SHAP and Deep SHAP approximation methods. First, we compared the computational efficiency and accuracy of Kernel SHAP vs. LIME and Shapley sampling values. Second, we designed user studies to compare SHAP values with alternative feature importance allocations represented by Deep LIFT and LIME. As might be expected, SHAP values prove more consistent with human intuition than other methods that fail to meet Properties 1-3 (Section 2). Finally, we use MNIST digit image classification to compare SHAP with Deep LIFT and LIME. |
| Researcher Affiliation | Academia | Scott M. Lundberg Paul G. Allen School of Computer Science University of Washington Seattle, WA 98105 slund1@cs.washington.edu Su-In Lee Paul G. Allen School of Computer Science Department of Genome Sciences University of Washington Seattle, WA 98105 suinlee@cs.washington.edu |
| Pseudocode | No | The paper refers to a 'full algorithm' in the supplementary material but does not include structured pseudocode or algorithm blocks within the main text. |
| Open Source Code | Yes | 1https://github.com/slundberg/shap |
| Open Datasets | Yes | Finally, we use MNIST digit image classification to compare SHAP with Deep LIFT and LIME. |
| Dataset Splits | No | The paper does not provide specific dataset split information (exact percentages, sample counts, citations to predefined splits, or detailed splitting methodology) needed to reproduce the data partitioning for training, validation, or testing. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers) needed to replicate the experiment. |
| Experiment Setup | Yes | SHAP and LIME were both run with 50k samples (Supplementary Figure 1); to improve performance, LIME was modified to use single pixel segmentation over the digit pixels. |