Robust Collaborative Learning with Linear Gradient Overhead
Authors: Sadegh Farhadkhani, Rachid Guerraoui, Nirupam Gupta, Lê-Nguyên Hoang, Rafael Pinot, John Stephan
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our theory by experiments on image classification and make our code available at https://github.com/LPD-EPFL/ robust-collaborative-learning. |
| Researcher Affiliation | Collaboration | 1EPFL, Lausanne, Switzerland. 2Tournesol, 3Calicarpa, Switzerland. Correspondence to: Sadegh Farhadkhani <sadegh.farhadkhani@epfl.ch>. |
| Pseudocode | Yes | Algorithm 1 MONNA as executed by a correct node i |
| Open Source Code | Yes | We validate our theory by experiments on image classification and make our code available at https://github.com/LPD-EPFL/ robust-collaborative-learning. |
| Open Datasets | Yes | We use the MNIST (Le Cun & Cortes, 2010) and CIFAR-10 (Krizhevsky et al., 2009) datasets, pre-processed as in (Baruch et al., 2019) and (El Mhamdi et al., 2021b). |
| Dataset Splits | No | The paper mentions using MNIST and CIFAR-10 datasets and discusses data heterogeneity, but it does not explicitly provide details about training, validation, and test splits (e.g., percentages or counts). |
| Hardware Specification | Yes | We list below the hardware components used: 1 Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz, 2 Nvidia Ge Force GTX 1080 Ti, 64 GB of RAM |
| Software Dependencies | Yes | Python 3.8.10 has been used to run our scripts. Besides the standard libraries associated with Python 3.8.10, our scripts use the following libraries: numpy 1.19.1, torch 1.6.0, torchvision 0.7.0, pandas 1.1.0, matplotlib 3.0.2, requests 2.21.0, urllib3 1.24.1, chardet 3.0.4, certifi 2018.08.24, pytz 2020.1, dateutil 2.6.1, pyparsing 2.2.0, cycler 0.10.0, kiwisolver 1.0.1, cffi 1.13.2. |
| Experiment Setup | Yes | For MNIST, we consider a convolutional neural network (CNN)... The model is trained using a learning rate γ = 0.75 for T = 600 iterations. ... For CIFAR-10, we use a CNN... we set γ = 0.5 and T = 2000 iterations. ... Momentum β = 0.99 (except for SCC: β = 0.9), Batch size b = 25 (MNIST), b = 50 (CIFAR-10). |