FaiREE: fair classification with finite-sample and distribution-free guarantee
Authors: Puheng Li, James Zou, Linjun Zhang
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we conduct experiments to test and understand the effectiveness of Fai REE. For both synthetic data and real data analysis, we compare Fai REE with the following representative methods for fair classification: Reject-Option-Classification (ROC) method in Kamiran et al. (2012), Eqodds-Postprocessing (Eq) method in Hardt et al. (2016), Calibrated Eqodds-Postprocessing (CEq) method in Pleiss et al. (2017) and Fair Bayes method in Zeng et al. (2022). The first three baselines are designed to cope with Equalized Odds and the last one is for Equality of Opportunity. |
| Researcher Affiliation | Academia | Puheng Li Peking University lphleo@pku.edu.cn James Zou Stanford University jamesz@stanford.edu Linjun Zhang Rutgers University linjun.zhang@rutgers.edu |
| Pseudocode | Yes | Algorithm 1: Fai REE for Equality of Opportunity ... Algorithm 2: Fai REE for Equalized Odds ... Algorithm 3: Fai REE for Demographic Parity ... Algorithm 4: Fai REE for Predictive Opportunity ... Algorithm 5: Fai REE for Equalized Accuracy |
| Open Source Code | Yes | Code is available at https://github.com/lph Leo/Fai REE |
| Open Datasets | Yes | In this section, we apply Fai REE to a real data set, Adult Census dataset (Dua et al., 2017), whose task is to predict whether a person s income is greater than $50,000. ... First, we apply Fai REE to German Credit dataset Kamiran & Calders (2009)... Then, we apply Fai REE to Compas Score dataset Angwin et al. (2016)... |
| Dataset Splits | Yes | The protected attribute is gender, and the sample size is 45,222, including 32561 training samples and 12661 test samples. To facilitate the numerical study, we randomly split data into training set, calibration set and test set at each repetition and repeat for 500 times. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models or memory used for experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | Fai REE takes input as 1). a given fairness guarantee G, such as Equality of Opportunity or Equalized Odds; 2). an error bound α, which controls the violation with respect to our given fairness notion; 3). a small tolerance level δ, which makes sure our final classifier satisfies our requirement with probability at least 1 δ; 4). a dataset S. ... α / / / 0.08 0.12 0.16 (Table 2) |