Explaining the Uncertain: Stochastic Shapley Values for Gaussian Process Models
Authors: Siu Lun Chau, Krikamol Muandet, Dino Sejdinovic
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our illustrations demonstrate the effectiveness of the proposed approach. Our illustrations demonstrate the effectiveness of GP-SHAP, Bayes GP-SHAP, and the Shapley prior for predictive explanations are provided in Section 5 and we conclude the paper in Section 6. In Section 5.1, we demonstrate the differences between model predictive uncertainty and estimation uncertainty captured in the stochastic Shapley values. For this purpose, we used the California housing dataset [41] from the Statl Lib repository... We trained our GP model using 25% and 100% of the data and calculated local stochastic explanations from GP-SHAP, Bayes SHAP, and Bayes GP-SHAP using 50% and 100% of coalitions. The results, shown in Figure 1, demonstrate that the magnitude of Bayes SHAP uncertainties (green bars) are uniform across features as it is designed to capture the overall estimation performance and not feature-specific uncertainty. |
| Researcher Affiliation | Academia | Siu Lun Chau CISPA Helmholtz Center for Information Security 66123 Saarbrücken, Germany siu-lun.chau@cispa.de Krikamol Muandet CISPA Helmholtz Center for Information Security 66123 Saarbrücken, Germany muandet@cispa.de Dino Sejdinovic School of Computer and Mathematical Sciences University of Adelaide Adelaide, South Australia, 5005 Australia dino.sejdinovic@adelaide.edu.au |
| Pseudocode | Yes | Algorithm 1 GP-SHAP / Bayes GP-SHAP |
| Open Source Code | Yes | Our code is included in the supplementary material, and we provide implementation details in the appendix. |
| Open Datasets | Yes | For this purpose, we used the California housing dataset [41] from the Statl Lib repository, which includes 20640 instances and 8 numerical features... To this end, we use Bayes GP-SHAP to explain a Gaussian process model trained on the breast cancer dataset [42]... We use the diabetes dataset [47] from the UCI repository, which contains 442 patients with 10 numerical features and the goal is to predict disease progression. |
| Dataset Splits | No | The paper explicitly mentions training and testing splits (e.g., 'Next, we feed 70% of the explanations to our predictive model as training data and the remaining 30% as test data.'), but does not specify a separate validation dataset split. |
| Hardware Specification | Yes | All illustrations are run locally on a Macbook Pro 2021 with Apple M1 pro chip. |
| Software Dependencies | No | The paper mentions software like 'Tree SHAP and Deep SHAP from the shap package [2]' and a 'sparse Variational GP approach', but it does not provide specific version numbers for these software components or libraries, which is required for reproducibility. |
| Experiment Setup | Yes | To fit the GP model, we employ a sparse Variational GP approach with 200 learnable inducing point locations. The evidence lower bound is optimized using batch gradient descent with a batch size of 64, a learning rate of 0.01, and 100 iterations. The RBF kernel with learnable bandwidths initialized using the median heuristic approach is used for the sparse GP. The inducing locations are initialized using a standard clustering approach... We repeat this process over 10 seeds... |