Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds
Authors: Hongyi Zhang, Sashank J. Reddi, Suvrit Sra
NeurIPS 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We implement Riemannian SVRG for PCA, and use the code for VR-PCA in [29]. Analytic forms for exponential map and parallel transport on hypersphere can be found in [1, Example 5.4.1; Example 8.1.1]. We conduct well-controlled experiments comparing the performance of two algorithms. Specifically, to investigate the dependence of convergence on δ, for each δ = 10 3/k where k = 1, . . . , 25, we generate a d n matrix Z = (z1, . . . , zn) where d = 103, n = 104 using the method Z = UDV > where U, V are orthonormal matrices and D is a diagonal matrix, as described in [29]. Note that A has the same eigenvalues as D2. All the data matrices share the same U, V and only differ in δ (thus also in D). We also fix the same random initialization x0 and random seed. We run both algorithms on each matrix for 50 epochs. For every five epochs, we estimate the number of epochs required to double its accuracy 2. This number can serve as an indicator of the global complexity of the algorithm. |
| Researcher Affiliation | Academia | Hongyi Zhang Sashank J. Reddi Carnegie Mellon University |
| Pseudocode | Yes | Algorithm 1: RSVRG (x0, m, , S) and Algorithm 2: GD-SVRG(x0, m, , S, K) |
| Open Source Code | No | The paper does not provide an explicit statement or link for the open-source code of the methodology described in the paper. |
| Open Datasets | No | The paper mentions generating datasets for experiments (e.g., "We generate 100 × 100 random PSD matrices using the Matrix Mean Toolbox [6]") but does not provide concrete access information (link, DOI, repository) for the generated datasets used. |
| Dataset Splits | No | The paper does not explicitly provide specific training/validation/test dataset splits needed to reproduce the experiments. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware used (e.g., GPU/CPU models, memory) to run its experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies used in the experiments. |
| Experiment Setup | Yes | We set = 1 100n, and choose m = n in Algorithm 1 for RSVRG, and use suggested parameters from [38] for other algorithms. ... for each δ = 10 3/k where k = 1, . . . , 25, we generate a d n matrix Z = (z1, . . . , zn) where d = 103, n = 104 ... We also fix the same random initialization x0 and random seed. We run both algorithms on each matrix for 50 epochs. |