Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Robustness Auditing for Linear Regression: To Singularity and Beyond
Authors: Ittai Rubinstein, Samuel Hopkins
ICLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present an efficient algorithm for certifying the robustness of linear regressions to removals of samples. We implement our algorithm and run it on several landmark econometrics datasets with hundreds of dimensions and tens of thousands of samples, giving the first non-trivial certificates of robustness to sample removal for datasets of dimension 4 or greater. We prove that under distributional assumptions on a dataset, the bounds produced by our algorithm are tight up to a 1 + o(1) multiplicative factor. We evaluate our algorithms experimentally and in theory. Experimental Results For comparison with prior works, we focus our experiments on ksign(e). We provide lower bounds on ksign(e) for benchmark datasets drawn from important studies in economics and social sciences... |
| Researcher Affiliation | Academia | Ittai Rubinstein & Samuel B. Hopkins Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology Cambridge, MA, USA EMAIL |
| Pseudocode | Yes | Algorithm 1: ACRE (Algorithm for Certifying Robustness Efficiently)... Algorithm 2: Refined Triangle Inequality... Algorithm 4: The OHARE Algorithm... Algorithm 5: KU Triangle Inequality... Algorithm 6: Dynamic Programming 2D... Algorithm 7: Spectral Bound Algorithm... Algorithm 8: Greedy Algorithm for Lower Bound and Candidate Set |
| Open Source Code | Yes | An implementation of our algorithms is available via Github. |
| Open Datasets | Yes | We study real-world datasets corresponding to each of the parameter estimation use-cases listed above: Nightlights Martinez (2022) (correlation controlled for additional features), Cash Transfer Angelucci & De Giorgi (2009) (randomized control trial), and the Oregon Health Insurance Experiment (OHIE) Finkelstein et al. (2012) (IV regression), with 14 distinct linear or instrumental-variables regressions drawn from the corresponding papers, all of which appeared in top econometrics journals. |
| Dataset Splits | No | Problem 1 (Robustness Auditing for OLS Regression). Given a linear regression instance (X1, Y1), . . . , (Xn, Yn) Rd+1, a direction e Rd, and an integer k n, what is k(e) = max S [n] |S|=n k β[n] βS, e (1)... Robustness auditing addresses the question: How would f have differed if we had been missing a small fraction of the data? The paper focuses on robustness to sample removal, which is not equivalent to specifying explicit training, validation, or test dataset splits in the conventional machine learning sense. |
| Hardware Specification | Yes | Our implementation is efficient enough to run our algorithms with n up to 3 104 and d up to 103 on a single CPU core with < 64GB of RAM. |
| Software Dependencies | No | An implementation of our algorithms is available via Github. The paper indicates where the code is hosted but does not list any specific software dependencies or their versions (e.g., Python, PyTorch, specific libraries). |
| Experiment Setup | Yes | We removed these bad samples for the sake of our analysis (both to avoid regressing over a clearly problematic dataset, and also because not doing so would cause the regression to include nearly empty categories in several different one-hot encodings which is beyond the scope of the OHARE algorithm). This had only a minor effect on the regression results. To overcome this, we replace the hectacres (amount of land owned) column with log (hectacres + median hectacres). We then reran the regression and robustness analysis to see the effects of this change to produce the results in Table 1 and Figure 2. |