Feature Importance Disparities for Data Bias Investigations
Authors: Peter W Chang, Leor Fishman, Seth Neel
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In Section 5, we conduct a thorough empirical evaluation, auditing for large FID subgroups on the Student (Cortez & Silva, 2008), COMPAS (Angwin et al., 2016), Bank (Moro et al., 2014), and Folktables (Ding et al., 2021) datasets, using LIME, SHAP, saliency maps, and linear regression coefficient as feature importance notions. |
| Researcher Affiliation | Academia | 1SAFR AI Lab, Harvard Business School & Kempner Institute, Boston, MA 2Harvard College, Cambridge, MA. Correspondence to: Seth Neel <sethneel.ai@gmail.com>. |
| Pseudocode | Yes | Algorithm 1 Iterative Constrained Optimization |
| Open Source Code | Yes | The code used for our experiments is available at github.com/safr-ai-lab/xai-disparity. |
| Open Datasets | Yes | We used four popular datasets for the experiments: Student (Cortez & Silva, 2008), COMPAS (Angwin et al., 2016), Bank (Moro et al., 2014), and Folktables (Ding et al., 2021). |
| Dataset Splits | No | Datasets were split into 80 20 train-test split except for Student which was split 50 50 due to its small size. |
| Hardware Specification | Yes | The optimization was run using GPU computing on NVIDIA Tesla V100s. |
| Software Dependencies | No | The parameter vector θ for a logistic regression classifier was randomly initialized with a Py Torch random seed of 0 for reproducability. We used an ADAM (Kingma & Ba, 2015) optimizer with a learning rate of .05 as our heuristic solver for the loss function. |
| Experiment Setup | Yes | We empirically found that setting the hyperparameter B = 10^4 µ(fj) worked well on all datasets... We set the learning rate for exponentiated gradient descent to η = 10^-5... We found that setting the error tolerance hyperparameter ν = .05 µ(fj) n αL worked well... For all datasets and methods we ran for at most T = 5000 iterations... We used an ADAM (Kingma & Ba, 2015) optimizer with a learning rate of .05... Empirical testing found that values of λs = 10^5 and λc = 10^-1 returned appropriate subgroup sizes... |