Robust and Actively Secure Serverless Collaborative Learning
Authors: Nicholas Franzese, Adam Dziedzic, Christopher A. Choquette-Choo, Mark R Thomas, Muhammad Ahmad Kaleem, Stephan Rabanser, Congyu Fang, Somesh Jha, Nicolas Papernot, Xiao Wang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical evaluation focuses on exploring three major axes: (1) the Byzantine robustness of our implementations due to modifications we introduced, (2) the computational efficiency of our protocol, and (3) the tradeoff between computational efficiency and Byzantine robustness. |
| Researcher Affiliation | Collaboration | 1Northwestern University, 2University of Toronto and Vector Institute, 3Google, 4University of Wisconsin-Madison, 5CISPA |
| Pseudocode | Yes | Figure 3: Main protocol outline for the malicious setting. Figure 7: P2P Learning with RSA. Figure 8: Centered Box Clipping. Figure 9: FLTrust. |
| Open Source Code | No | The paper does not include a statement about releasing source code for the described methodology or a link to a code repository. |
| Open Datasets | Yes | In Figure 4, we use the IID MNIST dataset and 20 peers, of which there are 10 malicious workers5. [...] MNIST (Digits) and EMNIST (Letters) datasets were used as the datasets with the data being evenly divided among the peers. |
| Dataset Splits | No | The paper mentions training and testing but does not explicitly provide details for a validation split (e.g., percentages, sample counts, or methodology for a separate validation set). |
| Hardware Specification | Yes | We used an m5.metal instance on Amazon EC2 to obtain the benchmarks reported in Figure 5. |
| Software Dependencies | No | To benchmark accuracy and robustness, and we used the NTL package [40] in C++ to implement the local computation for the aggregation steps of our malicious-secure framework. The Py Torch [36] framework was used for all experiments. The paper mentions software names like PyTorch and NTL but does not provide specific version numbers for these or other relevant libraries. |
| Experiment Setup | Yes | Assuming a 10% adversarial corruption threshold (i.e. setting p = 1/10), we obtain a committee size of 46. We use this committee size for experiments with RSA and CC. With FLTrust, in order to accommodate secret share multiplications with Shamir secret sharing, we guarantee Pr[X n/3] < 2 40, which gives a committee size of 121. Here θ is a public constant large enough to limit discretization error of local updates during scaling in the present study we set θ to 32 in order to align with 32-bit fixed-point numbers. In our benchmarks for FLT, we set θ to 16 to compensate for the increased memory demands of this protocol. During training, each client uses a local mini-batch of size 32 at each round and a learning-rate of 0.01. The training experiments were repeated over two random seeds. |