Distributed Online Convex Optimization with Compressed Communication
Authors: Zhipeng Tu, Xi Wang, Yiguang Hong, Lei Wang, Deming Yuan, Guodong Shi
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical experiments are given to validate the theoretical findings and illustrate that the proposed algorithms can effectively reduce the total transmitted bits for distributed online training compared with the uncompressed baseline. |
| Researcher Affiliation | Academia | 1Academy of Mathematics and Systems Science, Chinese Academy of Sciences, China 2Australian Center for Field Robotics, School of AMME, The University of Sydney, Australia 3Department of Control Science and Engineering, Tongji University, China 4College of Control Science and Engineering, Zhejiang University, China 5School of Automation, Nanjing University of Science and Technology, China tuzhipeng@amss.ac.cn, wangxi14@mails.ucas.ac.cn, yghong@iss.ac.cn lei.wangzju@zju.edu.cn, dmyuan1012@gmail.com, guodong.shi@sydney.edu.au |
| Pseudocode | Yes | Algorithm 1 Distributed Online Gradient Descent with Difference Compression (DC-DOGD) [...] Algorithm 2 Distributed Online One-point Bandit Gradient Descent with Difference Compression (DC-DOBD) [...] Algorithm 3 Distributed Online Two-point Bandit Gradient Descent with Difference Compression (DC-DO2BD) |
| Open Source Code | Yes | The codes are provided in the supplementary materials. |
| Open Datasets | Yes | We adopt diabetes-binary-BRFSS2015 dataset with 70692 instances, 21 features, and 2 labels from Kaggle.2 Here, ai,j Rd with d = 21, and bi,j { 1, 1}. We standardize the data samples and distribute them evenly among N agents under the sorted setting, i.e., each agent only gets data samples from one class. The data set is from https://www.kaggle.com/code/encode0/diabetes-prediction-and-risk-factors-evaluation. |
| Dataset Splits | No | The paper mentions distributing the dataset among N agents but does not provide explicit details on traditional training, validation, and test dataset splits (e.g., percentages or sample counts). |
| Hardware Specification | Yes | All experiments are performed on a 64-bit Windows platform with the Intel(R) Core(TM) i7-6850K 3.6Ghz CPU. |
| Software Dependencies | No | The paper mentions using 'Network X' and 'scikit-learn' but does not specify their version numbers. |
| Experiment Setup | Yes | The parameters selection details are given in Appendix E. Specifically, ηt = D/(G√t) for convex losses and ηt = 1/(µt) for strongly convex losses. The exploration parameters for bandit feedbacks are set as ϵ = (1+4H)d BR 2 (T +c) −1 4 T −1 2 and ζ = ϵ/r for convex losses and ϵ = Hd2B2 ln(T +c) −1 3 and ζ = ϵ/r for strongly convex losses. For the full information feedback, the consensus stepsize γ is chosen as 0.2. For the one-point bandit feedback, γ is chosen as 0.1 for convex losses and 0.05 for strongly convex losses. For the two-point bandit feedback, γ is chosen as 0.2 for convex losses and 0.1 for strongly convex losses. The regularization parameter µ is 0.001. We fix D = 10, G = 10, B = 10, l = 10. |