Stochastic Spectral and Conjugate Descent Methods
Authors: Dmitry Kovalev, Peter Richtarik, Eduard Gorbunov, Elnur Gasanov
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our first experiment we study how the practical behavior of SSCD (Algorithm 4) depends on the choice of k. What we study here does not depend on the dimensionality of the problem (n), and hence it suffices to perform the experiments on small dimensional problems (n = 30). In this experiment we consider the regime of clustered eigenvalues described in the introduction and summarized in Table 3. In particular, we construct a synthetic matrix A R30 30 with the smallest 15 eigenvalues clustered in the interval (5, 5 + ) and the largest 15 eigenvalues clustered in the interval (θ, θ + ). |
| Researcher Affiliation | Academia | Dmitry Kovalev1,2 Eduard Gorbunov1 Elnur Gasanov1,2 Peter Richtárik2,3,1 1Moscow Institute of Physics and Technology, Dolgoprudny, Russia 2King Abdullah University of Science and Technology, Thuwal, Saudi Arabia 3University of Edinburgh, Edinburgh, United Kingdom |
| Pseudocode | Yes | Algorithm 1 Stochastic Descent (SD); Algorithm 2 Stochastic Spectral Descent (SSD); Algorithm 3 Randomized Coordinate Descent (RCD); Algorithm 4 Stochastic Spectral Coordinate Descent (SSCD) |
| Open Source Code | No | The paper does not contain an explicit statement offering open-source code for the methodology or a link to a code repository. |
| Open Datasets | No | The paper states, 'In this experiment we consider the regime of clustered eigenvalues described in the introduction and summarized in Table 3. In particular, we construct a synthetic matrix A R30 30...' and 'In Figure 3 we report on an experiment using a synthetic problem with data matrix A of dimension n = 105 (i.e., potentially with 1010 entries).' This indicates the use of synthetic data, not a publicly available dataset with concrete access information. |
| Dataset Splits | No | The paper uses synthetic data and does not describe specific train/validation/test dataset splits, which are typically found with real-world datasets. |
| Hardware Specification | No | The paper states, 'As all experiments were done on a laptop,' which is too general and lacks specific hardware details such as GPU/CPU models, processor types, or memory amounts. |
| Software Dependencies | No | The paper does not specify any software names with version numbers that would be necessary for replication. |
| Experiment Setup | Yes | In particular, we construct a synthetic matrix A R30 30 with the smallest 15 eigenvalues clustered in the interval (5, 5 + ) and the largest 15 eigenvalues clustered in the interval (θ, θ + ). We vary the tightness parameter and the separation parameter θ, and study the performance of SSCD for various choices of k. |