Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Towards Robust and Reliable Algorithmic Recourse
Authors: Sohini Upadhyay, Shalmali Joshi, Himabindu Lakkaraju
NeurIPS 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental evaluation on multiple synthetic and real-world datasets demonstrates the efficacy of the proposed framework. We evaluated our approach ROAR on real world data from financial lending and education domains... |
| Researcher Affiliation | Academia | Sohini Upadhyay Harvard University EMAIL Shalmali Joshi Harvard University EMAIL Himabindu Lakkaraju Harvard University EMAIL |
| Pseudocode | Yes | Algorithm 1 Our Optimization Procedure |
| Open Source Code | No | The paper does not provide concrete access to source code (specific repository link, explicit code release statement, or code in supplementary materials) for the methodology described in this paper. |
| Open Datasets | Yes | Our first dataset is the widely used and publicly available German credit dataset [8] from the UCI repository. Our second dataset is the Small Business Administration (SBA) case dataset [17]. Our last dataset contains student performance records of 649 students from two Portuguese secondary schools, Gabriel Pereira (GP) and Mousinho da Silveira (MS) [8, 6]. |
| Dataset Splits | Yes | We use 5-fold cross validation throughout our real world and synthetic experiments. On D1, we use 4 folds to train predictive models and the remaining fold to generate and evaluate recourses. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper only mentions 'binary cross entropy loss and the Adam optimizer' without providing specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | Our framework, ROAR, has the following parameters: the set of acceptable perturbations (defined in practice by δmax) and the tradeoff parameter λ. In our experiments on evaluating robustness to real world shifts, we choose δmax = 0.1 given that continuous features are scaled to zero mean and unit variance. We use binary cross entropy loss and the Adam optimizer to operationalize our framework, ROAR. |