Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime
Authors: Amit Attia, Matan Schliserman, Uri Sherman, Tomer Koren
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We study population convergence guarantees of stochastic gradient descent (SGD) for smooth convex objectives in the interpolation regime, where the noise at optimum is zero or near zero. We establish that after T steps of SGD on beta-smooth convex loss functions with stepsize 0 < eta < 2/beta, the last iterate exhibits expected excess risk e O(1 / (eta * (2 - beta * eta))^T * (1 - beta * eta)/2 + eta * (2 - beta * eta)^2 * T * beta * eta/2 * sigma^2). In particular, for a well-tuned stepsize we obtain a near optimal e O(1/T + sigma / T) rate for the last iterate, extending the results of Varre et al. [2021] beyond least squares regression; and when sigma = 0 we obtain a rate of O(1/ T) with eta = 1/beta, improving upon the best-known O(T^1/4) rate recently established by Evron et al. [2025] in the special case of realizable linear regression. |
| Researcher Affiliation | Collaboration | Amit Attia Matan Schliserman Uri Sherman Blavatnik School of Computer Science and AI Tel Aviv University EMAIL Tomer Koren Blavatnik School of Computer Science and AI Tel Aviv University and Google Research EMAIL |
| Pseudocode | Yes | Algorithm 1 Regularized Continual Classification Initialize x1 = 0. For each iteration t = 1, . . . ,T: Sample a dataset St uniformly from {Sj}m j=1 (with or without replacement). xt+1 arg min x Rd (z,y) St e yxTz+ lambda Algorithm 2 Projections onto Convex Sets (POCS) 1: Initialize: x1 = 0 2: For t = 1 to T: 3: Sample tau(t) Unif([m]). 4: xt+1 P tau(t) (xt) arg minx Ctau(t) x xt 5: Output xT |
| Open Source Code | No | The paper does not include experiments. |
| Open Datasets | No | The paper describes theoretical problem setups (e.g., "stochastic optimization problem", "empirical risk minimization objective") and applications (e.g., "continual linear regression", "Kaczmarz method"), but does not specify any particular dataset by name, provide links, DOIs, or formal citations for data access. The NeurIPS checklist confirms: "The paper does not include experiments." |
| Dataset Splits | No | The paper is theoretical and does not present experimental results using specific datasets, therefore, there is no mention of dataset splits. The NeurIPS checklist confirms: "The paper does not include experiments." |
| Hardware Specification | No | The paper is theoretical and does not present experimental results, therefore, no hardware specifications are mentioned. The NeurIPS checklist confirms: "The paper does not include experiments." |
| Software Dependencies | No | The paper is theoretical and does not present experimental results, therefore, no software dependencies with version numbers are mentioned. The NeurIPS checklist confirms: "The paper does not include experiments." |
| Experiment Setup | No | The paper is theoretical and does not present experimental results, therefore, no experimental setup details like hyperparameters or training schedules are provided. The NeurIPS checklist confirms: "The paper does not include experiments." |