Distributed Primal-Dual Optimization for Non-uniformly Distributed Data
Authors: Minhao Cheng, Cho-Jui Hsieh
IJCAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show significant improvements over existing methods when data is distributed non-uniformly. And even when data is partitioned randomly, our algorithm is still faster than all the competing methods due to the flexibility of setting K weights instead of one single weight. In this section, we combine the Block-wise step size selection algorithm with Stochastic line search and propose it as Stochastic Block LS algorithm. We use SDCA as our local solver and apply each method to the binary hinge-loss support vector machines. We include the following algorithms into comparison: Stochastic Block LS: our proposed algorithm, using 1% of training samples to conduct stochastic line search on duality gap, and q = 10 2n for approximating M. Co Co A [Jaggi et al., 2014]: they set the step size η = 1/K. Co Co A+ [Ma et al., 2015]: they slightly changed the local subproblems and used a constant step size to synchronize the updates.. BQO [Lee and Roth, 2015]: conducted an exact line search on dual objective function. We set τ = 0.001 as suggested in the original paper. We consider three datasets: rcv1, epsilon, covtype. Details are shown in Table 1. |
| Researcher Affiliation | Academia | Minhao Cheng1, Cho-Jui Hsieh 1,2 1 Department of Computer Science, University of California, Davis 2 Department of Statistics, University of California, Davis |
| Pseudocode | Yes | Algorithm 1: Distributed Primal-Dual Optimization Framework |
| Open Source Code | No | The paper does not contain any explicit statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | We consider three datasets: rcv1, epsilon, covtype. Details are shown in Table 1. Since epsilon and covtype doesn t have the testing dataset. We split 87.5% data instances to training set and 12.5% to testing set. |
| Dataset Splits | No | The paper mentions splitting data into training and testing sets, but does not explicitly describe a separate validation split or its size/methodology. It states: |
| Hardware Specification | Yes | In our experiments, we use 8-32 nodes in the TACC Stampede cluster where each node is with 256GB memory. |
| Software Dependencies | No | To have a fair comparison between all algorithms, we implement all methods in C++ using MPI-LIBLINEAR setting. The paper mentions C++ and MPI-LIBLINEAR but does not provide specific version numbers for these or any other software components. |
| Experiment Setup | Yes | We use SDCA as our local solver and apply each method to the binary hinge-loss support vector machines. We include the following algorithms into comparison: Stochastic Block LS: our proposed algorithm, using 1% of training samples to conduct stochastic line search on duality gap, and q = 10 2n for approximating M. Co Co A [Jaggi et al., 2014]: they set the step size η = 1/K. Co Co A+ [Ma et al., 2015]: they slightly changed the local subproblems and used a constant step size to synchronize the updates.. BQO [Lee and Roth, 2015]: conducted an exact line search on dual objective function. We set τ = 0.001 as suggested in the original paper. We test the algorithms using the default λ values used in previous papers, as listed in the Table 1. Also, results with different λ values are showed in the as well. |