Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Beyond Benign Overfitting in Nadaraya-Watson Interpolators
Authors: Daniel Barzilai, Guy Kornowski, Ohad Shamir
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical experiments complement our theory, demonstrating the same phenomena. [...] 6 Experiments In this section, we provide numerical simulations that illustrate and complement our theoretical findings. |
| Researcher Affiliation | Academia | Daniel Barzilai Weizmann Institute of Science EMAIL Guy Kornowski Weizmann Institute of Science EMAIL Ohad Shamir Weizmann Institute of Science EMAIL |
| Pseudocode | No | The paper describes theoretical proofs and analyses but does not include any explicit pseudocode or algorithm blocks. The methods are described in mathematical notation. |
| Open Source Code | No | 5. Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: The experiments described in Section 6 are very easy to reproduce based on our accurate description. We believe there is no added value by sharing this code. |
| Open Datasets | Yes | Next, we consider an experiment in which the data consists of images of handwritten 0 and 1 digits from the MNIST dataset. [...] Pope et al. [45]. |
| Dataset Splits | No | In all experiments, we sample m datapoints according to some distribution, flip each label independently with probability p, and plot the clean test error of ˆhβ for various values of β. [...] In Figure 5, on the left we plot the results with respect to the entire training set m = 12,665 and various values of p, and on the right we fix p = 0.1 and vary m. The paper describes generating data for synthetic experiments and using the 'entire training set' for MNIST, but it does not specify explicit train/test/validation splits with percentages, sample counts, or references to predefined splits for measuring the reported 'clean test error'. |
| Hardware Specification | No | 8. Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [No] Justification: We do not explicitly mention these, as the experiments are extremely lightweight and take at most a couple of minutes to run locally on a standard computer. |
| Software Dependencies | No | The paper does not explicitly mention any specific software libraries, frameworks, or programming languages with version numbers required to replicate the experiments. |
| Experiment Setup | Yes | In all experiments, we sample m datapoints according to some distribution, flip each label independently with probability p, and plot the clean test error of ˆhβ for various values of β. We ran each experiment 50 times, and plotted the average error surrounded by a 95% confidence interval. [...] For m = 2000 and various values of p [...] we fix p = 0.04 and vary m. [...] with m = 2000 and various values of p, and on the right we fix p = 0.04 and vary m. [...] with respect to the entire training set m = 12,665 and various values of p, and on the right we fix p = 0.1 and vary m. |