A Universal Catalyst for First-Order Optimization
Authors: Hongzhou Lin, Julien Mairal, Zaid Harchaoui
NeurIPS 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the Catalyst acceleration on three methods that have never been accelerated in the past: SAG [24], SAGA [6], and MISO-Prox. We focus on ℓ2-regularized logistic regression, where the regularization parameter µ yields a lower bound on the strong convexity parameter of the problem. We use three datasets used in [14], namely real-sim, rcv1, and ocr, which are relatively large, with up to n = 2 500 000 points for ocr and p = 47 152 variables for rcv1. |
| Researcher Affiliation | Academia | Hongzhou Lin1, Julien Mairal1 and Zaid Harchaoui1,2 1Inria 2NYU {hongzhou.lin,julien.mairal}@inria.fr zaid.harchaoui@nyu.edu |
| Pseudocode | Yes | Algorithm 1 Catalyst input initial estimate x0 Rp, parameters κ and α0, sequence (εk)k 0, optimization method M; |
| Open Source Code | No | The paper does not contain any statements about releasing open-source code for the methodology or provide links to a code repository. |
| Open Datasets | Yes | We use three datasets used in [14], namely real-sim, rcv1, and ocr, which are relatively large, with up to n = 2 500 000 points for ocr and p = 47 152 variables for rcv1. |
| Dataset Splits | No | The paper states it uses datasets 'real-sim, rcv1, and ocr' but does not specify any training, validation, or test splits (e.g., '80/10/10 split' or specific sample counts for each). |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments, such as exact GPU or CPU models, or memory specifications. |
| Software Dependencies | No | The paper mentions 'Python' in Appendix E regarding implementation details but does not provide specific version numbers for Python or any other key software libraries or solvers used in the experiments. |
| Experiment Setup | Yes | We compare MISO, SAG, and SAGA with their default parameters, which are recommended by their theoretical analysis (step-sizes 1/L for SAG and 1/3L for SAGA), and study several accelerated variants. The values of κ and ρ and the sequences (εk)k 0 are those suggested in the previous sections, with η=0.1 in (10). Other implementation details are presented in Appendix E. |