Beating SGD Saturation with Tail-Averaging and Minibatching
Authors: Nicole Muecke, Gergely Neu, Lorenzo Rosasco
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This section provides an empirical illustration to the effects characterized in the previous sections. We focus on two aspects of our results: the benefits of tail-averaging over uniform averaging as a function of the smoothness parameter r, and the impact of tail-averaging on the best choice of minibatch sizes. All experiments are conducted on synthetic data with d = 1, 000 dimensions, generated as follows. |
| Researcher Affiliation | Academia | Nicole Mücke Institute for Stochastics and Applications University of Stuttgart nicole.muecke@mathematik.uni-stuttgart.de Gergely Neu Universitat Pompeu Fabra gergely.neu@gmail.com Lorenzo Rosasco Universita degli Studi di Genova Istituto Italiano di Tecnologia Massachusetts Institute of Technology lrosasco@mit.edu |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described in this paper. |
| Open Datasets | No | The paper uses synthetic data generated as follows: 'We set Σ as a diagonal matrix with entries Σii = i 1/ν and choose w = Σre, where e is a vector of all 1 s. The covariates Xt are generated from a Gaussian distribution with covariance Σ, and labels are generated as Yt = w , Xt + εt, where εt is standard Gaussian noise.' No publicly available dataset is mentioned with access information. |
| Dataset Splits | No | The paper does not provide specific dataset split information (exact percentages, sample counts, citations to predefined splits, or detailed splitting methodology) needed to reproduce the data partitioning. It uses synthetic data for illustration. |
| Hardware Specification | No | The paper does not provide specific hardware details used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers) needed to replicate the experiment. |
| Experiment Setup | Yes | In our first experiment... We fix b = 1 and set γ = n 2r+ν 2r+1+ν as recommended in Corollary 1. In our second experiment... We fix r = 1/2 and set T = n/b for all tested values of b, amounting to a single pass over the data. |