Optimal Learning for Multi-pass Stochastic Gradient Methods
Authors: Junhong Lin, Lorenzo Rosasco
NeurIPS 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, simple numerical simulations are given in Section 4 to complement our theoretical results. In order to illustrate our theoretical results and the error decomposition, we first performed some simulations on a simple problem. Finally, we tested the simple SGM, mini-batch SGM, and batch GM, using similar step-sizes as those in the first simulation, on the Breast Cancer data-set. |
| Researcher Affiliation | Academia | Junhong Lin LCSL, IIT-MIT, USA junhong.lin@iit.it Lorenzo Rosasco DIBRIS, Univ. Genova, ITALY LCSL, IIT-MIT, USA lrosasco@mit.edu |
| Pseudocode | Yes | Algorithm 1. Let b [m]. Given any sample z, the b-minibatch stochastic gradient method is defined by ω1 = 0 and ωt+1 = ωt ηt 1 b i=b(t 1)+1 ( ωt, xji H yji)xji, t = 1, . . . , T, (4) where {ηt > 0} is a step-size sequence. Here, j1, j2, , jb T are independent and identically distributed (i.i.d.) random variables from the uniform distribution on [m] 1. |
| Open Source Code | No | No explicit statement or link providing concrete access to source code for the methodology described in this paper was found. |
| Open Datasets | Yes | Finally, we tested the simple SGM, mini-batch SGM, and batch GM, using similar step-sizes as those in the first simulation, on the Breast Cancer data-set 5. 5https://archive.ics.uci.edu/ml/datasets/ |
| Dataset Splits | No | The paper mentions training and testing sets for the Breast Cancer dataset in Figure 2, but does not provide specific details on the dataset splits (e.g., percentages, sample counts, or methodology for splitting) for reproduction. No explicit mention of a validation set split. |
| Hardware Specification | No | No specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments were provided. |
| Software Dependencies | No | No specific ancillary software details (e.g., library or solver names with version numbers like Python 3.8, CPLEX 12.4) needed to replicate the experiment were provided. |
| Experiment Setup | Yes | In the first experiment, we run mini-batch SGM, where the mini-batch size b = m, and the step-size ηt = 1/(8 m). In the second experiment, we run simple SGM where the step-size is fixed as ηt = 1/(8m), while in the third experiment, we run batch GM using the fixed step-size ηt = 1/8. |