Momentum Ensures Convergence of SIGNSGD under Weaker Assumptions
Authors: Tao Sun, Qingsong Wang, Dongsheng Li, Bao Wang
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct experiments on training Res Net110 from (He et al., 2016) for CIFAR-100 (Krizhevsky et al., 2009) classification using SIGNSGD and SIGNSGD-SIM with various batch sizes. |
| Researcher Affiliation | Academia | 1College of Computer, National University of Defense Technology, Hunan, China. 2University of Utah. |
| Pseudocode | Yes | Algorithm 1 SIGNSGD with SImple Momentum (SIGNSGD-SIM) |
| Open Source Code | No | Our code is based on open-source libraries2. 2github.com/akamaster/pytorch_resnet_ cifar10, github.com/epfml/error-feedback-SGD - The text states their code is based on open-source libraries and provides links to those libraries, but does not explicitly state their own implementation code is open-source or provide a link to it. |
| Open Datasets | Yes | We train various Res Net models from (He et al., 2016) on CIFAR-10/CIFAR-100 (Krizhevsky et al., 2009) |
| Dataset Splits | No | Both datasets are split into a training set of 50,000 images and a test set of 10,000 images. The paper mentions training and test sets, and data distribution for clients, but does not explicitly describe a validation set or its split. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory, or cloud instance types) used for running experiments were provided. |
| Software Dependencies | No | Our code is based on open-source libraries2. 2github.com/akamaster/pytorch_resnet_ cifar10, github.com/epfml/error-feedback-SGD - The paper mentions libraries but does not provide specific version numbers for them. |
| Experiment Setup | Yes | The learning rate is decimated twice during this time, first at 100 epochs and again at 150 epochs. The initial learning rate for a batch size of 128 is 1 10 3. (...) The momentum parameter of SIGNSGD-SIM is set to 0.9, and the weight decay for both algorithms is set to 1 10 4. |