Distributed Stochastic Gradient Descent with Event-Triggered Communication
Authors: Jemin George, Prudhvi Gurram7169-7178
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Results indicate that all agents report similar performance that is also comparable to the performance of a centrally trained neural network, while the event-triggered communication provides significant reduction in inter-agent communication. Results also show that the proposed algorithm allows the individual agents to classify the images even though the training data corresponding to all the classes are not locally available to each agent. |
| Researcher Affiliation | Collaboration | Jemin George,1 Prudhvi Gurram2,1 1CCDC Army Research Laboratory Adelphi, MD 20783 jemin.george.civ@mail.mil 2Booz Allen Hamilton Mc Lean, VA 22102 gurram prudhvi@bah.com |
| Pseudocode | Yes | Pseudo-code of the proposed distributed event-triggered SGD is given in Algorithm 1 (see supplementary material). |
| Open Source Code | No | The paper mentions a link to the PyTorch ResNet-20 implementation (https://github.com/akamaster/pytorch_resnet_cifar10) for a model used, but does not provide a link or statement for the authors' own source code for the DETSGRAD algorithm or experiments. |
| Open Datasets | Yes | MNIST data set is a handwritten digit recognition data set containing 60000 grayscale images of 10 digits (0-9) for training and 10000 images are used for testing. (http://yann.lecun.com/exdb/mnist/) ... CIFAR-10 data set is an image classification data set containing 50000 color images of 10 classes (...) for training and 10000 images are used for testing. (https://www.cs.toronto.edu/ kriz/cifar.html) |
| Dataset Splits | Yes | MNIST data set is a handwritten digit recognition data set containing 60000 grayscale images of 10 digits (0-9) for training and 10000 images are used for testing. ... Each agent aims to train its own neural network... each agent has access to 6000 training images, randomly sampled from the entire training set... each agent has access to the images corresponding to a single class... CIFAR-10 data set is an image classification data set containing 50000 color images of 10 classes (...) for training and 10000 images are used for testing. ... Each agent trains its own neural network... each agent has access to 6250 training images, randomly sampled from the entire training set. |
| Hardware Specification | No | The paper does not specify any hardware details such as CPU/GPU models, memory, or specific computing environments used for the experiments. |
| Software Dependencies | No | The paper mentions using LeNet-5 and ResNet-20 models, and links to a PyTorch implementation for ResNet, but does not provide specific version numbers for software dependencies like PyTorch, TensorFlow, or other libraries used for implementation. |
| Experiment Setup | Yes | Here we select αk = 0.1 (εk+1) and βk = 0.2525 (εk+1)1/10 , where ε = 10 5 for Distributed SGD and DETSGRAD. We select αk = 0.001 (εk+1) for centralized SGD. ... For the DETSGRAD experiments, we select the broadcast event trigger threshold υ0 = 0.2 Nparameters... The final test accuracies of the 10 agents after 40 training epochs... Here we select αk = 0.1 (εk+1) and βk = 0.2525 (εk+1)1/10 , where ε = 0.00025 for Distributed SGD and DETSGRAD. We select αk = 0.1 (εk+1), where ε = 10 5 for centralized SGD. For the DETSGRAD experiment, we select the broadcast event trigger threshold υ0 = 0.01 Nparameters... The final test accuracies of the 8 agents after 200 training epochs... |