Near-Optimal Solutions of Constrained Learning Problems
Authors: Juan Elenter, Luiz F. O. Chamon, Alejandro Ribeiro
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 EXPERIMENTAL VALIDATION To illustrate the theoretical results from Sections 3 and 4, we return to the counterfactually fair learning problem from Example 2.1. We work with the COMPAS dataset, where the task is to predict recidivism while remaining insensitive to the protected variables gender and race, which can take the values [ Male , Female ] and [ African American , Hispanic , Caucasian , Other ] respectively. We take the parametrized model fθ to be a 2-layer NN with sigmoid activations, so that the resulting constrained learning problem is non-convex. Further experimental details are provided in Appendix A.16. We compare the accuracy and constraint satisfaction of three models: an unconstrained predictor, trained without any additional constraints; a last iterate predictor, corresponding to the final iterate fθ(T) of an empirical version of Algorithm 1; and a randomized predictor that samples a model uniformly at random from the sequence of primal iterates {fθ(t))}T t=t0 for each prediction. As shown in Fig. 2 (Left), the unconstrained model is slightly better than the two constrained ones in terms of predictive accuracy. This advantage comes at the cost of less counterfactually fair predictions, i.e., a model more sensitive to the protected features (Fig. 2, Middle). The key point of this experiment, however, is that the last iterate and randomized predictors provide similar accuracy and constraint satisfaction, as predicted by Theorem 3.1. Additionally, Fig. 2 (Right) showcases the impact of the parametrization richness on the constraint violation of last primal iterates. |
| Researcher Affiliation | Academia | Juan Elenter University of Pennsylvania Luiz F. O. Chamon University of Stuttgart Alejandro Ribeiro University of Pennsylvania |
| Pseudocode | Yes | Algorithm 1 Dual Constrained Learning |
| Open Source Code | No | The paper does not contain any explicit statements about releasing its source code or provide links to a code repository for the methodology described. |
| Open Datasets | Yes | We work with the COMPAS dataset, where the task is to predict recidivism while remaining insensitive to the protected variables gender and race, which can take the values [ Male , Female ] and [ African American , Hispanic , Caucasian , Other ] respectively. and Consider the COMPAS study (Pro Publica, 2020), with the goal of predicting recidivism based on past offense data while controlling for gender and racial bias. |
| Dataset Splits | No | The paper mentions using the COMPAS dataset and details training parameters such as batch size and number of iterations, but it does not provide explicit percentages, sample counts, or references to predefined train, validation, or test dataset splits. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments, such as GPU models, CPU specifications, or cloud computing instance types. |
| Software Dependencies | No | The paper mentions using 'ADAM' for training but does not provide specific software dependencies with version numbers, such as programming languages, libraries, or frameworks (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | We adopt the same data pre-processing steps as in (Chamon & Ribeiro, 2020) and use a two-layer neural network with 64 nodes and sigmoid activations. The counterfactual fairness constraint upper bound is set to 0.001. We train this model over T = 400 iterations using a ADAM, with a batch size of 256, a primal learning rate equal to 0.1 and weight decay magnitude set to 10 4. The dual variable learning rate is set to 2. |