Certification of Distributional Individual Fairness
Authors: Matthew Wicker, Vihari Piratla, Adrian Weller
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section we empirically validate our proposed method on a variety of datasets. We first describe the datasets and metrics used. We then cover a wide range of experimental ablations and validate our bounds on real-world distribution shifts. We conclude the section with a study of how IF training impacts other notions of fairness.1 |
| Researcher Affiliation | Academia | Matthew Wicker The Alan Turing Institute mwicker@turing.ac.uk Vihari Piratia University of Cambridge vp421@cam.ac.uk Adrian Weller University of Cambridge & The Alan Turing Institute |
| Pseudocode | No | The paper does not contain any explicitly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | Code to reproduce experiments can be found at: https://github.com/matthewwicker/DistributionalIndividualFairness |
| Open Datasets | Yes | We benchmark against the Adult, Credit, and German datasets from the UCI dataset repository (Dua & Graff, 2017). The German or Satlog dataset predicts credit risk of individuals. The Credit dataset predicts if an individual will default on their credit. The Adult dataset predicts if an individuals income is greater than 50 thousand dollars. We additionally consider three datasets from the Folktables datasets, Income, Employ and Coverage, which are made up of millions of data points curated from the 2015 to 2021 US census data (Ding et al., 2021). |
| Dataset Splits | No | The paper states 'Unless stated otherwise, we use δ = 0.05, γ = 0.1, and with respect to 1000 test-set individuals.' but does not provide specific percentages or sample counts for training, validation, and test splits needed for reproducibility, nor does it refer to standard predefined splits. |
| Hardware Specification | No | The paper mentions that 'the results from this paper were run on a laptop' and 'timings from (Benussi et al., 2022) use a multi-GPU machine,' but it does not provide specific hardware details such as GPU/CPU models, memory amounts, or precise machine specifications. |
| Software Dependencies | No | The paper does not provide specific software dependencies or library versions (e.g., Python 3.x, PyTorch 1.x) used for implementation or experimentation. |
| Experiment Setup | Yes | Unless stated otherwise, we use δ = 0.05, γ = 0.1, and with respect to 1000 test-set individuals. Complete experimental details are given in Appendix A. |