Distributed Principal Component Analysis with Limited Communication
Authors: Foivos Alimisis, Peter Davies, Bart Vandereycken, Dan Alistarh
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our approach experimentally, comparing the proposed method of Riemannian gradient quantization against three other benchmark methods: Full-precision Riemannian gradient descent: Riemannian gradient descent, as described in Section 3.2, is performed with the vectors communicated at full (64-bit) precision. ... We show convergence results (Figure 1) for the methods on four real datasets: Human Activity from the MATLAB Statistics and Machine Learning Toolbox, and Mice Protein Expression, Spambase, and Libras Movement from the UCI Machine Learning Repository [9]. |
| Researcher Affiliation | Collaboration | Foivos Alimisis Department of Mathematics University of Geneva Peter Davies Department of Computer Science University of Surrey Bart Vandereycken Department of Mathematics University of Geneva Dan Alistarh IST Austria & Neural Magic, Inc. |
| Pseudocode | Yes | We present now our main algorithm, which is inspired by quantized gradient descent firstly designed by [22], and its similar version in [3]. 1. Choose an arbitrary machine to be the master node, let it be i0. 2. Choose x(0) Sd 1 (we analyze later specific ways to do that). 3. Consider the following parameters ... For t 0: 8. Take a gradient step using the exponential map: x(t+1) = expx(t)( ηq(t)) with step-size η (the step-size is discussed later). In Tx(t+1)Sd 1:... |
| Open Source Code | Yes | Our code is publicly available 1. 1https://github.com/IST-DASLab/QRGD |
| Open Datasets | Yes | We show convergence results (Figure 1) for the methods on four real datasets: Human Activity from the MATLAB Statistics and Machine Learning Toolbox, and Mice Protein Expression, Spambase, and Libras Movement from the UCI Machine Learning Repository [9]. |
| Dataset Splits | No | No, the paper mentions datasets but does not provide specific details on how they were split into training, validation, or test sets, nor does it refer to standard predefined splits for these purposes. |
| Hardware Specification | No | No, the paper does not provide any specific details about the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | No, the paper does not specify any software dependencies (e.g., libraries, frameworks, or operating systems) with their version numbers. |
| Experiment Setup | No | No, while some algorithmic parameters are defined symbolically, the paper does not provide concrete numerical values for hyperparameters or system-level training configurations needed for reproduction. |