Incentivizing Honesty among Competitors in Collaborative Learning and Optimization
Authors: Florian E. Dorner, Nikola Konstantinov, Georgi Pashaliev, Martin Vechev
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Lastly, we empirically demonstrate the effectiveness of our incentive scheme on a standard non-convex federated learning benchmark. Our work shows that explicitly modeling the incentives and actions of dishonest clients, rather than assuming them malicious, can enable strong robustness guarantees for collaborative learning. ... We simulate Fed SGD Mc Mahan et al. (2017) with clients corrupting their messages to different degrees, and record how players rewards and penalties are affected by their aggressiveness α for different penalty constants C. ... For both experiments, we randomly split writers into two groups A and B containing one and two thirds of the writers respectively, and corrupt the gradient estimates ... We record both the cross-entropy loss f achieved by the final model θT on the test set, as well as the sum of the squared deviations mi t mt 2 incurred by each individual client i across all steps. |
| Researcher Affiliation | Academia | Florian E. Dorner MPI for Intelligent Systems, Tübingen ETH Zurich florian.dorner@tuebingen.mpg.de Nikola Konstantinov INSAIT, Sofia University nikola.konstantinov@insait.ai Georgi Pashaliev Sofia High School of Mathematics Martin Vechev ETH Zurich martin.vechev@inf.ethz.ch |
| Pseudocode | No | The paper describes algorithmic steps in text but does not include formal pseudocode or algorithm blocks. |
| Open Source Code | No | The paper references using 'pytorch Paszke et al. (2019)' and 'the network used in the LEAF repository 3' and 'We downloaded the Fe MNIST and Twitter datasets using the code provided at https://github.com/Talwalkar Lab/leaf/tree/master/data/femnist'. This indicates usage of existing open-source resources, but there is no explicit statement or link providing the authors' own implementation code for the methodology described in this paper. |
| Open Datasets | Yes | Lastly, we empirically demonstrate the effectiveness of our incentive scheme on a standard non-convex federated learning benchmark. ... we conduct experiments on the Fe MNIST and Twitter datasets from LEAF (Caldas et al., 2018)... We downloaded the Fe MNIST and Twitter datasets using the code provided at https://github.com/Talwalkar Lab/leaf/tree/master/data/femnist |
| Dataset Splits | No | The paper states: 'We test on the remaining 10% of the data.' implying a train/test split, but no explicit mention of a separate validation set or its split details. |
| Hardware Specification | Yes | All experiments were conducted using a single GPU each4 per run. 4We used assigned GPUs from a cluster that employs mostly, but not exclusively Nvidia V100 GPUs |
| Software Dependencies | No | The paper mentions 'pytorch Paszke et al. (2019)' but does not specify a version number for PyTorch or any other software libraries. |
| Experiment Setup | Yes | We simulate Fed SGD Mc Mahan et al. (2017)... to train a CNN-classifier using the architecture provided by Caldas et al. (2018) for the Fe MNIST dataset... We train a two-layer linear classifier with 384 hidden neurons on top of frozen 'bert-base-uncased' embeddings on the Twitter Sentiment Analysis dataset... update our neural network s parameters θt with learning rate 0.06... The network we train is based on the network used in the LEAF repository ... It consists of two convolutional layers with relu activations, kernel size 5, (2, 2) padding and 32 and 64 filters, respectively, each followed by max pooling with kernel size and stride 2. After the convolutional layers, there is a single hidden dense layer with 2048 neurons and a relu activation, and a dense output layer. |