Certifiably Byzantine-Robust Federated Conformal Prediction
Authors: Mintong Kang, Zhen Lin, Jimeng Sun, Cao Xiao, Bo Li
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we demonstrate the robustness of Rob-FCP against various portions of malicious clients under multiple Byzantine attacks on five standard benchmark and real-world healthcare datasets. We empirically evaluate Rob-FCP against multiple Byzantine attacks. |
| Researcher Affiliation | Collaboration | 1University of Illinois at Urbana-Champaign, USA 2Carle s Illinois College of Medicine, USA 3GE Healthcare, USA 4University of Chicago, USA. |
| Pseudocode | Yes | An overview of Rob FCP is presented in Figure 2, with the pseudocode detailed in Algorithm 1 in Appendix F. Algorithm 1 Malicious client identification |
| Open Source Code | Yes | The codes to reproduce all the evaluation results are publicly available at https://github.com/ kangmintong/Rob-FCP. |
| Open Datasets | Yes | We evaluate Rob-FCP on a variety of standard datasets, including MNIST (Deng, 2012), CIFAR-10 (Krizhevsky et al.), and Tiny-Image Net (Le & Yang, 2015). Our evaluation of Rob-FCP also cover two realistic healthcare datasets: the Sleep Heart Health Study (SHHS) dataset (Zhang et al., 2018) and a pathology dataset Path MNIST (Yang et al., 2023). |
| Dataset Splits | Yes | In this work, we focus on the split conformal prediction setting (Papadopoulos et al., 2002), where the data samples are randomly partitioned into two disjoint sets: a training set Itr and a calibration (hold-out) set Ical = [n]\Itr. For SHHS: 2,514 patients (2,545,869 samples) were used for training the DNN, and 2,514 patients (2,543,550 samples) were used for calibration and testing. |
| Hardware Specification | Yes | The valuation is done on a RTX A6000 GPU. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers for reproducibility. |
| Experiment Setup | Yes | Without specification, the desired coverage level 1 \alpha is set 0.9. By default, we set \beta to 0.5 to establish a consistent level of data heterogeneity. |