The Skellam Mechanism for Differentially Private Federated Learning
Authors: Naman Agarwal, Peter Kairouz, Ziyu Liu
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our theoretical findings and extensive experimental evaluations demonstrate that the Skellam mechanism provides the same privacy-accuracy trade-offs as the continuous Gaussian mechanism, even when the precision is low. Our theoretical findings and extensive experimental evaluations demonstrate that the Skellam mechanism provides the same privacy-accuracy trade-offs as the continuous Gaussian mechanism, even when the precision is low. |
| Researcher Affiliation | Collaboration | Google Research Carnegie Mellon University {namanagarwal, kairouz}@google.com, ziyuliu@cs.cmu.edu |
| Pseudocode | Yes | Algorithm 1 summarizes the aggregation procedure for the distributed Skellam mechanism via secure aggregation as well as the parameters used for the experiments. |
| Open Source Code | Yes | Our method is implemented in Tensor Flow Privacy [32] and Tensor Flow Federated [24] and will be open-sourced.2 [2] https://github.com/google-research/federated/tree/master/distributed_dp |
| Open Datasets | Yes | Setup We evaluate on three public federated datasets with real-world characteristics: Federated EMNIST [16], Shakespeare [31, 14], and Stack Overflow next word prediction (SO-NWP [8]). |
| Dataset Splits | No | The paper mentions client batch size and client learning rates, and that clients train for 1 epoch on their local datasets. It also states that for EMNIST and Shakespeare, the average test accuracy is reported over the last 100 rounds. However, it does not provide specific percentages or counts for training, validation, and test splits for reproducibility. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, processor types, or memory amounts used for running experiments. |
| Software Dependencies | No | Our experiments are implemented in Python, Tensor Flow Privacy [32], and Tensor Flow Federated [24]. While the software names are mentioned, specific version numbers for Python or the TensorFlow libraries are not provided. |
| Experiment Setup | Yes | For EMNIST, we follow [25] and fix c = 0.03, n = 100, T = 1500, client learning rate client = 0.32, server learning rate server = 1, and client batch size m = 20. For Shakespeare, we follow [6] and fix n = 100, T = 1200, client = 1, server = 0.32, and m = 4, and we sweep c ∈ {0.25, 0.5}. For SO-NWP, we follow [27] and fix c = 0.3, n = 100, T = 1600, client = 0.5, and m = 16, and we sweep server ∈ {0.3, 1} and limit max examples per client to 256. In all cases, clients train for 1 epoch on their local datasets, and the client updates are weighted uniformly (as opposed to weighting by number of examples). |