Stochastic Block BFGS: Squeezing More Curvature out of Data
Authors: Robert Gower, Donald Goldfarb, Peter Richtarik
ICML 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical tests on large-scale logistic regression problems reveal that our method is more robust and substantially outperforms current state-of-the-art methods. |
| Researcher Affiliation | Academia | Robert M. Gower GOWERROBERT@GMAIL.COM Donald Goldfarb GOLDFARB@COLUMBIA.EDU Peter Richt arik PETER.RICHTARIK@ED.AC.UK |
| Pseudocode | Yes | Algorithm 1 Stochastic Block BFGS Method; Algorithm 2 Block L-BFGS Update (Two-loop Recursion); Algorithm 3 Block L-BFGS Update (Factored loop recursion) |
| Open Source Code | Yes | All the code for the experiments can be downloaded from www.maths.ed.ac.uk/ prichtar/i software.html. |
| Open Datasets | Yes | We tested seven empirical risk minimization problems with a logistic loss and L2 regularizer using data from LIBSVM (Chang & Lin, 2011). |
| Dataset Splits | No | The paper does not explicitly provide specific training/validation/test dataset splits (e.g., percentages, sample counts, or explicit files). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running experiments. |
| Software Dependencies | No | All the methods were implemented in MATLAB. (Does not specify version number for MATLAB or other dependencies). |
| Experiment Setup | Yes | We set the regularization parameter λ = 1/n for all experiments. We set the subsampling size |St| = n throughout our tests. We tested each method with a stepsize η {100, 5 · 10−1, 10−1, . . . , 10−6, 5 · 10−7, 10−7} for the best outcome, and used the resulting η. Finally, we used m = n/|St| for the number of inner iterations... We set the memory to 10 for the MNJ method in all tests... |