Shapley explainability on the data manifold
Authors: Christopher Frye, Damien de Mijolla, Tom Begley, Laurence Cowton, Megan Stanley, Ilya Feige
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the effectiveness of these methods on higher-dimensional data with experiments. ... In this section, we perform additional experiments to study the performance and stability of Sec. 4.1 s methods, as well as their effectiveness on higher-dimensional data. |
| Researcher Affiliation | Industry | Christopher Frye, Damien de Mijolla, Tom Begley, Laurence Cowton, Megan Stanley, Ilya Feige Faculty, 54 Welbeck Street, London, UK |
| Pseudocode | No | The paper does not include any pseudocode or explicitly labeled algorithm blocks. |
| Open Source Code | No | The paper mentions using the SHAP package and sklearn, but does not provide a statement or link for the open-sourcing of their own methodology's code. |
| Open Datasets | Yes | We can demonstrate this on UCI Census Income data (Dua & Graff, 2017). ... We can demonstrate this on UCI Drug Consumption data (Dua & Graff, 2017). ... We can demonstrate this on real data using UCI Abalone data (Dua & Graff, 2017). ... For binary MNIST (Le Cun & Cortes, 2010), we trained a fully connected network... |
| Dataset Splits | No | For each experiment in this paper, we tuned hyperparameters to minimise the MSE of Eq. (16) on a held-out validation set; see App. B for numerical details. ... We split the dataset between 99% inliers and 1% outliers, with the classes generated according to: ... The paper mentions 'test set accuracy' and 'held-out validation set' but does not specify the exact percentages or sample counts for the train/validation/test splits for the main datasets (UCI, MNIST). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions using 'sklearn parameters' and the 'SHAP package', and references 'Adam (Kingma & Ba, 2015)' for optimization, but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | Yes | All neural networks in this paper used 2 flat hidden layers, Adam (Kingma & Ba, 2015) for optimisation, and a batch size of 256. ... We scanned over a grid with hidden layer size = {128, 256, 512} learning rate = {10-3, 10-4} choosing the point with minimal MSE on a held-out validation set after 10k epochs of training; see Table 2. ... Table 2: Optimal hyperparameters found for computing on-manifold Shapley values. |