Subgroup Robustness Grows On Trees: An Empirical Baseline Investigation
Authors: Josh Gardner, Zoran Popovic, Ludwig Schmidt
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct an empirical comparison of several previously-proposed methods for fair and robust learning alongside state-of-the-art tree-based methods and other baselines. Via experiments with more than 340,000 model configurations on eight datasets, we show that tree-based methods have strong subgroup robustness, even when compared to robustnessand fairness-enhancing methods. |
| Researcher Affiliation | Collaboration | Josh Gardner1 Zoran Popovi c1 Ludwig Schmidt1,2 1 University of Washington 2 Allen Institute for AI |
| Pseudocode | No | No pseudocode or clearly labeled algorithm blocks were found in the paper. |
| Open Source Code | Yes | We provide code to reproduce our experiments, along with an interactive tool to explore the best-performing hyperparameter configurations, at https://github.com/jpgard/ subgroup-robustness-grows-on-trees. |
| Open Datasets | Yes | We evaluate the 17 models over eight datasets covering a variety of prediction tasks and domains. We use two binary sensitive attributes from each dataset, for a total of four nonoverlapping subgroups in each dataset. A summary of the datasets used in this work is given in Table 1. |
| Dataset Splits | No | The paper mentions extensive hyperparameter tuning across datasets, which implies the use of validation sets, but does not explicitly provide specific split percentages, sample counts, or detailed splitting methodology in the main body of the paper for reproducibility beyond referring to the datasets themselves. |
| Hardware Specification | No | The paper mentions '1 CPU-day' and '58 GPU-days' for training runs, but does not specify the exact CPU or GPU models used. It also mentions 'Hyak computing cluster' but without hardware specifics. |
| Software Dependencies | No | The paper mentions software like XGBoost, Light GBM, and scikit-learn, but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | For each model, we conduct a grid search over a large set of hyperparameters. We give the complete set of hyperparameters tuned for each model in Section F. |