Estimate Sequences for Variance-Reduced Stochastic Composite Optimization
Authors: Andrei Kulunchakov, Julien Mairal
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5. Experiments Following Bietti & Mairal (2017); Zheng & Kwok (2018) we consider logistic regression with Drop Out (Srivastava et al., 2014), which consists of randomly setting to zero each vector entry with probability δ, leading to the problem... We use three data sets alpha, ckn-cifar, and gene from different nature, which are presented in the appendix, along with other experimental details. Figure 1. Objective function value on a logarithmic scale with λ = 1/10n (left) and λ = 1/100n (right), with no Drop Out. |
| Researcher Affiliation | Academia | 1Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, 38000 Grenoble, France. |
| Pseudocode | Yes | Algorithm 1 Iteration (A) with random-SVRG estimator; Algorithm 2 Accelerated and robust random-SVRG |
| Open Source Code | No | The paper does not include an unambiguous statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | We consider logistic regression with Drop Out (Srivastava et al., 2014)... We use three data sets alpha, ckn-cifar, and gene from different nature, which are presented in the appendix, along with other experimental details. |
| Dataset Splits | No | The paper mentions using specific datasets but does not provide details on training, validation, or test splits (e.g., percentages or sample counts). |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments, such as GPU/CPU models or memory specifications. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers that would be needed for reproducibility. |
| Experiment Setup | Yes | Following Bietti & Mairal (2017); Zheng & Kwok (2018) we consider logistic regression with Drop Out (Srivastava et al., 2014)... The parameter λ acts as a lower bound on µ and we consider λ = 1/10n, which is of the order of the smallest value that one would try when doing parameter search. We use three data sets alpha, ckn-cifar, and gene... We use them always with their theoretical step size, except rand-SVRG, which we evaluate with η = 1/3L in order to obtain a fair comparison with acc-SVRG. When using the decreasing step size strategy, we add the suffix -d to the method s name, and we consider also a minibatch variant of acc-SGD, denoted by acc-mb-SGD with minibatch size b = p L/µ. We also use the initial step size 1/3L for rand-SVRG-d since it performs better in practice. |