A Near-Optimal Algorithm for Debiasing Trained Machine Learning Models
Authors: Ibrahim M. Alabdulmohsin, Mario Lucic
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically validate its advantages on standard benchmark datasets across both classical algorithms as well as modern DNN architectures and demonstrate that it outperforms previous post-processing methods while performing on par with in-processing.5 Experiments |
| Researcher Affiliation | Industry | Ibrahim Alabdulmohsin Google Research, Brain Team Zürich, Switzerland ibomohsin@google.com Mario Lucic Google Research, Brain Team Zürich, Switzerland lucic@google.com |
| Pseudocode | Yes | Algorithm 1: Pseudocode of the Randomized Threshold Optimizer (RTO). |
| Open Source Code | No | The paper does not contain an unambiguous statement of releasing the code for the described methodology or a direct link to a source-code repository. |
| Open Datasets | Yes | We evaluate performance on two real-world datasets, namely the Adult income dataset [Kohavi, 1996] and the Default of Credit Card Clients (DCCC) dataset [Yeh and Lien, 2009], both from the UCI Machine Learning Repository [Blake and Merz, 1998]. Celeb A dataset [Liu et al., 2015]. COCO dataset [Lin et al., 2014]. |
| Dataset Splits | Yes | Specifically, we split the data that was not used in the original classifier into three subsets of equal size: (1) training data for the post-processing rules, (2) validation for hyperparameter selection, and (3) test data. |
| Hardware Specification | No | The paper mentions running experiments on deep neural networks and various pretrained models but does not provide specific hardware details such as GPU/CPU models, processor types, or memory amounts. |
| Software Dependencies | No | The paper mentions using the 'Fair Learn software package [Dudik et al., 2020]' but does not provide specific version numbers for it or any other key software components like programming languages or libraries. |
| Experiment Setup | Yes | The value of the hyper-parameter θ of the ROC algorithm is chosen in the grid {0.01, 0.02, . . . , 1.0}. In the proposed algorithm, the parameter γ is chosen in the grid {0.01, 0.02, 0.05, 0.1, 0.2} while ρ is chosen in the gird E[y] {0, 0.05, 0.1}. All hyper-parameters are selected based on a separate validation dataset. We train four classifiers: (1) random forests with depth 10, (2) k-NN with k = 10, (3) a two-layer neural network with 128 hidden nodes, and (4) logistic regression whose parameter C is fine-tuned from a grid of values in a logarithmic scale between 10 4 and 104 using 10-fold cross validation. The learning rate in our algorithm is fixed to 10 1(K/T)1/2, where T is the number of steps, and ϵ = 0. In every task, we build a linear classifier on top of the pretrained features. Inspired by the Hyper Rule in [Kolesnikov et al., 2020], we train for 50 epochs with an initial learning rate of 0.003, which is dropped by factor of 10 after 20, 30, and 40 epochs. All images are resized to 224 224. For augmentation, we use random horizontal flipping and cropping, where we increase the dimension of the image to 248 248 before cropping an image of size 224 224 at random. |