Learning to Understand: Identifying Interactions via the Möbius Transform
Authors: Justin Kang, Yigit Efe Erginbas, Landon Butler, Ramtin Pedarsani, Kannan Ramchandran
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In addition to our asymptotic analysis, we provide synthetic and real-world experiments that verify that our algorithm performs well even in the finite n regime. Furthermore, our results are non-adaptive meaning that all samples can be computed in parallel. Code has been made publicly available 1. |
| Researcher Affiliation | Academia | Justin Singh Kang UC Berkeley justin kang@berkeley.edu Yigit Efe Erginbas UC Berkeley erginbas@berkeley.edu Landon Butler UC Berkeley landonb@berkeley.edu Ramtin Pedarsani UC Santa Barbara ramtin@ece.ucsb.edu Kannan Ramchandran UC Berkeley kannanr@berkeley.edu |
| Pseudocode | Yes | Algorithm 1 Sparse M obius Transform (SMT) |
| Open Source Code | Yes | Code has been made publicly available 1https://github.com/basics-lab/sparse Mobius Transform |
| Open Datasets | Yes | For a particular review, we define its function as a mapping from maskings of words (using the [UNK] token) to the model s logit value associated to the correct sentiment classification. For Figure 2, we use the first ten sentences in the dataset with 17, 18, or 19 words, where words separated through spaces in the review. Below, we include the reviews and their low degree and sparse approximations calculated with equations 20 and 22 respectively. |
| Dataset Splits | No | The paper does not explicitly specify training, validation, and test splits with percentages or sample counts. It describes the data used for experiments but not the specific partitioning for training/validation/testing. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or specific cloud instances) used for running the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers. It mentions tools like XGBoost, BERT, and SHAP, but not their versions or other ancillary software. |
| Experiment Setup | Yes | In Figure 7, we consider the four explanation models described below. Shapley Values: We approximate Shapley values by iterating through permutations of the inputs [2]. For an efficient implementation of the algorithm, we use the SHAP Python package [2]. To measure the faithfulness captured by Shapley values at some sparsity level r, we consider approximations that only include the top-r Shapley values by magnitude. Banzhaf Values: We approximate Banzhaf values using the Maximum Sample Reuse Monte Carlo procedure described in [31]. To measure the faithfulness captured by Banzhaf values at some sparsity level r, we consider approximations that only include the top-r Banzhaf values by magnitude. Faith-Banzhaf Indices: We calculate Faith-Banzhaf indices using the regression formulation described in [4]. To measure the faithfulness captured by sparse approximations of Faith-Banzhaf indices, we modify the regression problem by adding an ℓ1 penalty on the values of the Faith-Banzhaf indices. We vary the penalty coefficient to obtain different levels of sparsity. SMT: We run SMT (Algorithm 1) to obtain a sparse M obius representation ˆF with support supp( ˆF). Then, we fine-tune the values of the coefficients by solving the following regression problem over a uniformly sampled set of points D Zn 2: . To measure the faithfulness captured by sparse approximations, we modify the regression problem by adding an ℓ1 penalty on the values of the M obius coefficients. Then, we vary the penalty coefficient λ to obtain different levels of sparsity: |