Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Outside the Echo Chamber: Optimizing the Performative Risk
Authors: John P Miller, Juan C Perdomo, Tijana Zrnic
ICML 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We complement our theoretical findings with an empirical evaluation of different methods on two tasks: the strategic classification simulator from Perdomo et al. (2020), and a synthetic linear regression example. ... For each of the following experiments, we run each algorithm 50 times and display 95% bootstrap confidence intervals. |
| Researcher Affiliation | Academia | 1University of California, Berkeley. Correspondence to: John Miller <miller EMAIL>, Juan C. Perdomo <EMAIL>, Tijana Zrnic <EMAIL>. |
| Pseudocode | Yes | Algorithm 1 Two-Stage Algorithm for Location Families |
| Open Source Code | No | The paper does not include an unambiguous statement that the authors are releasing the code for the work described in this paper, nor does it provide a direct link to a source-code repository. |
| Open Datasets | Yes | We complement our theoretical findings with an empirical evaluation of different methods on two tasks: the strategic classification simulator from Perdomo et al. (2020), and a synthetic linear regression example. |
| Dataset Splits | No | The paper does not provide specific dataset split information (exact percentages, sample counts, citations to predefined splits, or detailed splitting methodology) needed to reproduce the data partitioning. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers like Python 3.8, CPLEX 12.4) needed to replicate the experiment. |
| Experiment Setup | Yes | We provide a formal description of all the procedures, as well as a detailed description of the experimental setup in Appendix D. Appendix D itself details parameters like 'x ~ N(0, ฮฃx), U_y ~ N(0, ฯ^2_y), y = ฮฒ^T x + ยต^T ฮธ + U_y' for linear regression, and 'add โ2-regularization to the logistic loss' for strategic classification. |