Feature Noise Induces Loss Discrepancy Across Groups
Authors: Fereshte Khani, Percy Liang
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our results on three real-world datasets: for predicting the final grade of secondary school students , final GPA of law students , and crime rates in the US communities , where the group g is either race or gender. |
| Researcher Affiliation | Academia | Fereshte Khani 1 Percy Liang 1 1Department of Computer Scinece, Stanford University. Correspondence to: Fereshte Khani <fereshte@stanford.edu>. |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Reproducibility. All code, data and experiments for this paper are available on the Coda Lab platform at https: //worksheets.codalab.org/worksheets/ 0x7c3fb3bf981646c9bc11c538e881f37e. |
| Open Datasets | Yes | We consider three real-world datasets from the fairness literature. See Table 2 for a summary and Appendix G for more details. ... for predicting the final grade of secondary school students (Cortez and Silva, 2008), final GPA of law students (Wightman and Ramsey, 1998), and crime rates in the US communities (Redmond and Baveja, 2002). |
| Dataset Splits | No | We run each experiment 100 times, each time randomly performing a 80 20 train-test split of the data, and reporting the average on the test set. This specifies only a train-test split, without explicit mention of a separate validation split. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU, CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., library names with specific versions) required to replicate the experiments. |
| Experiment Setup | Yes | We standardize all features and the target in all datasets (except the group membership feature) to have mean 0 and variance 1. We run each experiment 100 times, each time randomly performing a 80 20 train-test split of the data, and reporting the average on the test set. We compute the least squares estimator for each of the two observation functions: o g which only have access to non-group features, and o+g which have access to all features. We consider two types of noise: 1. Equal noise: for different values of σ2 u we add independent normal noise (u N(0, σ2 u)) to each feature except the group membership. 2. Omitting features: We start with a random order of the non-group features and omit features, which is nearly equivalent to adding normal noise with a very high variance (u N(0, 10000)) to them sequentially. |