Anticipating Performativity by Predicting from Predictions
Authors: Celestine Mendler-Dünner, Frances Ding, Yixin Wang
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically we show that given our identifiability conditions hold, standard variants of supervised learning that predict from predictions by treating the prediction as an input feature can find transferable functional relationships that allow for conclusions about newly deployed predictive models. Empirically, we demonstrate that supervised learning succeeds in finding MY even in finite samples. |
| Researcher Affiliation | Academia | Celestine Mendler-Dünner Max Planck Institute for Intelligent Systems, Tübingen cmendler@tuebingen.mpg.de; Frances Ding Univerity of California, Berkeley frances@berkeley.edu; Yixin Wang University of Michigan yixinw@umich.edu |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper includes a general statement in the checklist that code is provided for reproduction, but it does not provide a direct URL to a source-code repository for its own methodology or an explicit statement within the main text that its code is in supplementary material. The link provided in reference [11] is for a dataset used, not the paper's own code. |
| Open Datasets | Yes | We generated semi-synthetic data for our experiments, using a Census income prediction dataset from folktables.org [11].; Code available at: https://github.com/zykls/folktables. |
| Dataset Splits | No | The paper mentions generating a "training dataset" and a "test dataset" but does not explicitly state or describe a "validation" dataset or specific percentages for any splits. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments (e.g., GPU/CPU models, memory specifications). |
| Software Dependencies | No | The paper mentions general software or libraries like "scikit-learn" or neural networks, but it does not provide specific version numbers for any key software components or dependencies used in the experiments. |
| Experiment Setup | Yes | If not specified otherwise, fθ is fit to the original dataset to minimize squared error, while fφ is trained on randomly shuffled labels. The coefficients β are determined by linear regression on the original dataset. The hyperparameter α quantifies the performativity strength that we vary in our experiments. We optimize h SL in (8) over H being the class of linear functions. |