Individually Fair Rankings
Authors: Amanda Bower, Hamid Eftekhari, Mikhail Yurochkin, Yuekai Sun
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we demonstrate the efficacy of Sen STIR for learning individually fair LTR models. One key conclusion is that enforcing individual fairness is adequate to achieve group fairness but not vice versa. See Section B of the appendix for full details about the experiments. |
| Researcher Affiliation | Collaboration | Amanda Bower Department of Mathematics University of Michigan amandarg@umich.edu Hamid Eftekhari Department of Statistics University of Michigan hamidef@umich.edu Mikhail Yurochkin IBM Research MIT-IBM Watson AI Lab mikhail.yurochkin@ibm.com Yuekai Sun Department of Statistics University of Michigan yuekai@umich.edu |
| Pseudocode | Yes | Algorithm 1: Sen STIR: Sensitive Set Transport Invariant Ranking |
| Open Source Code | No | We use the implementation found at https://github.com/ashudeep/ Fair-PGRank for the synthetic and German experiments, whereas we use our own implementation for the Microsoft experiments because we could not get their code to run on this data. |
| Open Datasets | Yes | The German Credit data set (Dua & Graff, 2017) consists of 1000 individuals with binary labels indicating if they are credit worthy or not. We use the version of the German Credit data set that Singh & Joachims (2019) used found at https://www.kaggle.com/uciml/ german-credit. and Microsoft Learning to Rank The Microsoft Learning to Rank data set (Qin & Liu, 2013)... The data and feature descriptions can be found at https://www.microsoft.com/en-us/research/project/mslr/. |
| Dataset Splits | Yes | We use an 80/20 train/test split of the original 1000 data points (for German Credit) and We use Fold 1 s train/validation/test split. (for Microsoft Learning to Rank). |
| Hardware Specification | No | All experiments were ran a cluster of CPUS. We do not require a GPU. |
| Software Dependencies | No | We implement Sen STIR in Tensor Flow and use the Python POT package to compute the fair distance between queries and to compute Equation (3.4), which requires solving optimal transport problems. Throughout this section, variable names from our code are italicized, and the abbreviation we use to refer to these variables/hyperparameters are followed in parenthesis. |
| Experiment Setup | Yes | See Table 1 for the remaining values of hyperparameters where the column names have been defined in the previous section except for ϵ, which refers to ϵ in the definition of the fair regularizer. For synthetic data, the fair regularization strength ρ varied in {.0003, .001}. For German, ρ is varied in {.001, .01, 0.02, ...}. and For every experiment, all weights are initialized by picking numbers in [ .0001, .0001] uniformly at random, λ in Algorithm 1 is always initialized with 2, and the learning rate for Adam for the score function hθ is always .001. |