Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
A New Paradigm for Counterfactual Reasoning in Fairness and Recourse
Authors: Lucius E.J. Bynum, Joshua R. Loftus, Julia Stoyanovich
IJCAI 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experiments In this section, we demonstrate our criteria computationally on real and simulated data. |
| Researcher Affiliation | Academia | Lucius E.J. Bynum1 , Joshua R. Loftus2 and Julia Stoyanovich1,3 1New York University, Center for Data Science 2London School of Economics, Department of Statistics 3New York University, Tandon School of Engineering EMAIL, EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1 Backtracking Counterfactual Sampling |
| Open Source Code | Yes | Code to reproduce all our experiments is available online.5 Our code repository is located on Git Hub at: https://github.com/ lbynum/backtracking-counterfactual-fairness-and-recourse. |
| Open Datasets | Yes | Used to illustrate counterfactual fairness in [Kusner et al., 2017], the law school dataset from [Wightman, 1998] contains information on law students demographics and academic performance compiled from a 1998 Law School Admission Council survey. |
| Dataset Splits | No | The paper mentions 'datasets of size 500' and 'random sample of 5000 observations' but does not specify explicit training, validation, or test dataset splits. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., programming language versions, library versions, solver versions) used in the experiments. |
| Experiment Setup | Yes | We fit several predictors on datasets of size 500 from the above equations, each a linear regression using some subset of the available covariates (excluding A)." and "We generate data for Scenario 2 with a small modification to Scenario 1: A Bern(0.5), ZA N(A/2, 1), ZA N(0, 1), X1 N(2ZA + ZA , 1), X2 N(3ZA , 1), and Y N(X1 + X2 + 2ZA + ZA 2, 1). |