RSN: Randomized Subspace Newton
Authors: Robert Gower, Dmitry Kovalev, Felix Lieder, Peter Richtarik
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform numerical experiments which demonstrate the efficiency of our method as compared to accelerated gradient descent and the full Newton method. In this section we evaluate and compare the computational performance of RSN (Algorithm 1) on generalized linear models (26). Specifically, we focus on logistic regression, i.e., φi(t) = ln (1 + e yit) , where yi { 1, 1} are the target values for i = 1, . . . , n. Gradient descent (GD), accelerated gradient descent (AGD) [24] and full Newton methods6 are compared with RSN. We consider 6 datasets with a diverse number of features and samples (see Table 1 for details). |
| Researcher Affiliation | Academia | Robert M. Gower LTCI, T el ecom Paristech, IPP, France gowerrobert@gmail.com Dmitry Kovalev KAUST, Saudi Arabia dmitry.kovalev@kaust.edu.sa Felix Lieder Heinrich-Heine-Universit at D usseldorf, Germany lieder@opt.uni-duesseldorf.de Peter Richt arik KAUST, Saudi Arabia and MIPT, Russia peter.richtarik@kaust.edu.sa |
| Pseudocode | Yes | Algorithm 1 RSN: Randomized Subspace Newton |
| Open Source Code | No | The paper does not provide any concrete access to source code for the methodology described. |
| Open Datasets | Yes | Table 1: Details of the data sets taken from LIBSM [6] and Open ML [31]. [6] Chih Chung Chang and Chih Jen Lin. LIBSVM : A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3):1 27, April 2011. [31] Joaquin Vanschoren, Jan N. van Rijn, Bernd Bischl, and Luis Torgo. Openml: Networked science in machine learning. SIGKDD Explorations, 15(2):49 60, 2013. |
| Dataset Splits | No | The paper mentions using datasets for evaluation but does not provide specific dataset split information (e.g., percentages, sample counts, or methodology) for training, validation, or testing. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions using LIBSVM and OpenML, but it does not specify any software versions for libraries, frameworks, or other dependencies required to replicate the experiments. |
| Experiment Setup | Yes | For regularization we used λ = 10 10 and stopped methods once the gradients norm was below tol = 10 6 or some maximal number of iterations had been exhausted. To ensure fairness and for comparability purposes, all methods were supplied with the exact Lipschitz constants and equipped with the same line-search strategy (see Algorithm 3 in the supplementary material). |