Active Sampling for Min-Max Fairness
Authors: Jacob D Abernethy, Pranjal Awasthi, Matthäus Kleindessner, Jamie Morgenstern, Chris Russell, Jie Zhang
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present an empirical evaluation of both our proposed algorithms in Section 5. ... 5. Experiments ... Figure 2. Logistic regression on the Drug Consumption dataset. ... Figure 3. Logistic regression on the COMPAS dataset. ... Table 1. Logistic regression on the Diabetes dataset. |
| Researcher Affiliation | Collaboration | 1Georgia Tech, USA 2Google, USA 3Amazon Web Services, Germany 4University of Washington, USA. Correspondence to: J. Abernethy <prof@gatech.edu>, P. Awasthi <pranjalawasthi@google.com>, M. Kleindessner <matkle@amazon.de>, J. Morgenstern <jamiemmt@cs.washington.edu>, C. Russell <cmruss@amazon.de>, J. Zhang <claizhan@uw.edu>. |
| Pseudocode | Yes | Algorithm 1 A generic group-specific loss aware sampling strategy; Algorithm 2 Min-max Stochastic Gradient Descent; Algorithm 3 Accelerated Min-max Gradient Descent |
| Open Source Code | Yes | Code available on https://github.com/amazonresearch/active-sampling-for-minmaxfairness |
| Open Datasets | Yes | In Section 5 we use the Drug Consumption dataset (Fehrman et al., 2015) and the Diabetes 130-US Hospitals dataset (Diabetes dataset; Strack et al., 2014), which are both publicly available in the UCI repository (Dua and Graff, 2019). We also use the COMPAS dataset (Angwin et al., 2016), which is publicly available at https://github.com/propublica/ compas-analysis. |
| Dataset Splits | Yes | The Diabetes dataset contains 101766 records, which we split into a training, validation, and a held-out test set of equal size. ... We use the entire dataset for training (sampling and performing SGD updates) and for reporting performance metrics. We either use the entire dataset or a small subset comprising 20 datapoints sampled uniformly at random from each group as validation set (for determining the group with the highest loss). |
| Hardware Specification | No | The paper does not explicitly describe the hardware used for running the experiments. It only mentions general training procedures without specifying GPU, CPU, or cloud instance types. |
| Software Dependencies | No | We implemented Algorithm 2 based on Scikit-learn s (Pedregosa et al., 2011; https://scikit-learn.org) SGDClassifier class, and we implemented Algorithm 3 using Pytorch (https://pytorch.org/). When applying our strategy (Algorithm 1) to the MLP on the Diabetes dataset, we used Scikit-learn s MLPClassifier class. |
| Experiment Setup | Yes | For all experiments we use a mini-batch size of 32. ... we set the regularization parameter for l2-regularization to 10^-6 for the logistic regression classifier and to 10^-4 for the MLP classifier. ... For Algorithm 2, we used a learning rate of 0.01 on the Drug Consumption dataset and 0.005 (logistic regression) or 0.001 (MLP) on the Diabetes dataset. ... We initialized our classifier training on a subset of 2000 training points. |