Rest-Katyusha: Exploiting the Solution's Structure via Scheduled Restart Schemes
Authors: Junqi Tang, Mohammad Golbabaee, Francis Bach, Mike E. davies
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the effectiveness of our approach via numerical experiments. In this section we describe our numerical experiments on our proposed algorithm Rest-Katyusha (Alg.2) and also the adaptive Rest-Katyusha (Alg.3). Figure 1: Lasso Experiments on (A) Madelon and (B) RCV1 |
| Researcher Affiliation | Academia | Junqi Tang School of Engineering University of Edinburgh, UK J.Tang@ed.ac.uk Mohammad Golbabaee Department of Computer Science University of Bath, UK M.Golbabaee@bath.ac.uk Francis Bach INRIA ENS PSL Research University, France Francis.Bach@inria.fr Mike Davies School of Engineering University of Edinburgh, UK Mike.Davies@ed.ac.uk |
| Pseudocode | Yes | Algorithm 1 Katyusha (x0, m, S, L) Algorithm 2 Rest-Katyusha (x0, µc, S0, β, T, L) Algorithm 3 Adaptive Rest-Katyusha (x0, µ0, S0, β, T, L) |
| Open Source Code | No | The paper does not provide an explicit statement or link to open-source code for the described methodology. |
| Open Datasets | Yes | We run all the algorithms with their theoretical step sizes for Madelon and REGED dataset, while for RCV1 dataset we adopt minibatch scheme for all the algorithms and grid-search the step sizes which optimize these algorithms performance. Table 1: Datasets for the Experiments and Minibatch Size Choice for the Algorithms DATA SET SIZE (n, d) MINIBATCH REF. (A) MADELON (2000, 500) 1 [49] (B) RCV1 (20242, 47236) 80 [49] (C) REGED (500, 999) 1 [50] |
| Dataset Splits | No | The paper mentions using Madelon, RCV1, and REGED datasets for experiments but does not explicitly provide training/validation/test split percentages, sample counts, or detailed splitting methodology. It implies standard usage but does not specify its own splits for reproducibility. |
| Hardware Specification | No | The paper does not explicitly provide specific hardware details such as GPU/CPU models, memory, or cloud instance specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions algorithms like SVRG and Katyusha but does not specify any software dependencies or their version numbers used for implementation (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | In all our experiments we set β = 5 and S0 = S for convenience. We first do a grid-search on the estimate of µc for Rest-Katyusha which provides the best convergence performance... We run all the algorithms with their theoretical step sizes for Madelon and REGED dataset, while for RCV1 dataset we adopt minibatch scheme for all the algorithms and grid-search the step sizes which optimize these algorithms performance. For the adaptive Rest-Katyusha we fix our starting estimate of µc as 10e-5 throughout all the experiments. |