Generalized Disparate Impact for Configurable Fairness Solutions in ML
Authors: Luca Giuliani, Eleonora Misino, Michele Lombardi
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we discuss an empirical evaluation performed with three objectives: 1) testing how the kernel choice and fine-grained constraints affect the Ge DI semantics; 2) studying the relation with other metrics, in particular the DIDI and the HGR indicators; 3) investigating how the use of different ML models and constraint enforcement algorithms impacts effectiveness. |
| Researcher Affiliation | Academia | 1Department of Computer Science and Engineering, University of Bologna, Bologna, Italy. Correspondence to: Luca Giuliani <luca.giuliani13@unibo.it>, Eleonora Misino <eleonora.misino2@unibo.it>. |
| Pseudocode | No | The paper contains mathematical equations and descriptions of procedures, but no explicitly labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | Yes | The source code and the datasets are available at https://github.com/giuluck/General ized Disparate Impact under MIT license. |
| Open Datasets | Yes | We rely on two common benchmark datasets in the field of fair AI: Communities & Crimes1 with a continuous target, and Adult2 with a binary target. 1https://archive.ics.uci.edu/ml/datasets/Communities+ and+Crime 2https://archive.ics.uci.edu/ml/datasets/Adult |
| Dataset Splits | Yes | All these experiments are performed using a 5-fold cross-validation procedure on the entire dataset. |
| Hardware Specification | Yes | All the models are trained on a machine with an Intel Core I9 10920X 3.5G and 64GB of RAM. |
| Software Dependencies | Yes | The Random Forest and Gradient Boosting models are based on their available implementations in scikit-learn 1.0.2 with default parameters, while the Neural Network and the semantics-based Regularization models are implemented using torch 1.13.1. As regards Moving Target s optimization routine, we leverage the Python APIs offered by gurobipy 10.0 to solve it within our Python 3.7 environment. The backend solver is Gurobi 10.0, for which we use the default parameters except for Work Limit = 60. |
| Experiment Setup | Yes | Specifically, the hyper-parameters of neural-based models are obtained via a grid search analysis with train-test splitting on the two unconstrained tasks aimed at maximizing test accuracy. In particular, the neural models are trained for 200 epochs with batch size 128 and two layers of 256 units for the Communities & Crimes tasks and three layers of 32 units for the Adult tasks. The only exception is the semantics-based Regularization model which runs for 500 epochs to compensate the fact that it is trained full-batch in order to better deal with group constraints. Additionally, all the neurons have Re LU activation function except for the output one, which has either linear or sigmoid activation depending on whether the task is regression or classification, respectively. Accordingly, the loss function is either mean square error or binary crossentropy, but in both cases the training is performed using the Adam optimizer with default hyperparameters. |