Model Agnostic Supervised Local Explanations
Authors: Gregory Plumb, Denali Molitor, Ameet S. Talwalkar
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate, on several UCI datasets, that MAPLE is at least as accurate as random forests and that it produces more faithful local explanations than LIME, a popular interpretability system. and We present experiments demonstrating that: 1) MAPLE is generally at least as accurate as random forests, GBRT, and SILO 2) MAPLE provides faithful self-explanations, i.e., its local linear model at x is a good local explanation of the prediction at x 3) MAPLE is more accurate in predicting a black-box predictive model s response than a comparable and popular explanation system, LIME [17] 4) The local training distribution can be used to detect the presence of global patterns in the predictive model. |
| Researcher Affiliation | Academia | Gregory Plumb CMU gdplumb@andrew.cmu.edu Denali Molitor UCLA dmolitor@math.ucla.edu Ameet Talwalkar CMU talwalkar@cmu.edu |
| Pseudocode | No | The paper describes algorithms using mathematical formulas and prose, but there is no explicitly labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | Yes | 1The source code for our model and experiments is at https://github.com/GDPlumb/MAPLE. |
| Open Datasets | Yes | We run our experiments on several of the UCI datasets [7]. and the reference [7] Dua Dheeru and EfiKarra Taniskidou. UCI machine learning repository, 2017. |
| Dataset Splits | Yes | Each dataset was divided into a 50/25/25 training, validation, and testing split for each of the trials. |
| Hardware Specification | No | The paper does not mention any specific hardware used for running the experiments (e.g., GPU/CPU models, cloud instances). |
| Software Dependencies | No | The paper mentions 'scikit-learn' for SVR, but does not provide a specific version number, which is required for reproducibility. 'We use a Support Vector Regression (SVR) model (implementation and standard parameters from scikit-learn) as a black-box predictive model.' This is not enough for software_dependencies. |
| Experiment Setup | Yes | Each dataset was divided into a 50/25/25 training, validation, and testing split for each of the trials. All variables, including the response, were standardized to have mean zero and variance one. and We use our proposed causal metric defined in (1) as our evaluation metric, defining px as N(x, σI), using the squared l2 loss, and approximating the expectation by taking x from the testing set and drawing five x per testing point. We chose σ = 0.1 as a reasonable choice for the neighborhood scale because the data was normalized to have variance one. |