Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions
Authors: Hao Wang, Berk Ustun, Flavio Calmon
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our approach through experiments on real-world datasets, showing that it can repair different forms of disparity without a significant drop in accuracy. |
| Researcher Affiliation | Academia | 1Harvard University, MA, USA. Correspondence to: Hao Wang <hao_wang@g.harvard.edu>, Berk Ustun <berk@seas.harvard.edu>, Flavio P. Calmon <flavio@seas.harvard.edu>. |
| Pseudocode | Yes | Algorithm 1 Distributional Descent. |
| Open Source Code | Yes | Additional Resources We provide a software implementation of our tools at http://github.com/ustunb/ctfdist. |
| Open Datasets | Yes | To this end, we consider processed versions of the adult dataset (Bache & Lichman, 2013) and the Pro Publica compas dataset (Angwin et al., 2016). |
| Dataset Splits | Yes | For each dataset, we use: 30% of samples to train a classifier h(x) to repair; 50% of samples to recover a counterfactual distribution via Algorithm 1; 20% of samples as a hold-out set to evaluate the performance of the repaired model. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU, CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper mentions using 'ℓ2-logistic regression' and implies the use of related libraries, but it does not specify any software or library names with version numbers necessary for reproducibility. |
| Experiment Setup | Yes | We use ℓ2-logistic regression to train a classifier h(x) as well as the classifiers ˆy0(x) and ˆs(x) that we use to estimate the influence functions in Algorithm 1. We tune the parameters and estimate the performance of each classifiers using a standard 10-fold CV setup. |