Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Shapley Residuals: Quantifying the limits of the Shapley value for explanations
Authors: Indra Kumar, Carlos Scheidegger, Suresh Venkatasubramanian, Sorelle Friedler
NeurIPS 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In summary, we: ... show via a number of experiments that Shapley residuals capture meaningful information for model explanations in realistic scenarios (Section 6) and Having theoretically justified Shapley residuals in previous sections, we now focus on illustrating what these residuals can help us understand about models on a real-world dataset. |
| Researcher Affiliation | Academia | I. Elizabeth Kumar Department of Computer Science Brown University Providence, RI 02912 EMAIL Carlos Scheidegger Department of Computer Science University of Arizona Tucson, AZ 85721 EMAIL Suresh Venkatasubramanian Department of Computer Science Brown University Providence, RI 02912 EMAIL Sorelle A. Friedler Computer Science Dept. Haverford College Haverford, PA 19041 EMAIL |
| Pseudocode | Yes | Algorithm 1 Exactly calculate the ith Shapley value and Shapley residual of v |
| Open Source Code | Yes | Code is provided in the supplementary material |
| Open Datasets | Yes | Consider the Shapley values and residuals for an occupancy detection dataset3 (20,560 instances) used to predict whether an office room is occupied. 3https://archive.ics.uci.edu/ml/datasets/Occupancy+Detection+ |
| Dataset Splits | Yes | A decision tree model with maximum depth 3 is trained on 75% of the data using the features light and hour. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU models, CPU types, or memory specifications used for running experiments. |
| Software Dependencies | No | The paper mentions using "our own implementation of Kernel SHAP" but does not specify any software dependencies with version numbers (e.g., Python, PyTorch, specific library versions). |
| Experiment Setup | Yes | A decision tree model with maximum depth 3 is trained on 75% of the data using the features light and hour. and We then calculate the Shapley values and residuals (using 50 randomly sampled background rows from the test set) for 1000 randomly sampled test instances. |