Have it your way: Individualized Privacy Assignment for DP-SGD
Authors: Franziska Boenisch, Christopher Mühl, Adam Dziedzic, Roy Rinberg, Nicolas Papernot
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4 Empirical Evaluation For our empirical evaluation, we implement our methods in Python 3.9 and extend standard Opacus with our individualized privacy parameters and a per-privacy group accounting. We perform evaluation on the MNIST [23], SVHN [29], and CIFAR10 [22] dataset, using the convolutional architectures from Tramer and Boneh [38] for most experiments, and additionally evaluate on various language datasets and diverse (larger) model architectures in Section 4.2. To evaluate the utility of our methods, we use the datasets standard train-test splits and report test accuracies. |
| Researcher Affiliation | Academia | Franziska Boenisch boenisch@cispa.de CISPA Helmholtz Center for Information Security Christopher Mühl christopher.muehl@fu-berlin.de Free University Berlin Adam Dziedzic dziedzic@cispa.de CISPA Helmholtz Center for Information Security Roy Rinberg roy.rinberg@columbia.edu Columbia University Nicolas Papernot nicolas.papernot@utoronto.ca University of Toronto & Vector Institute |
| Pseudocode | Yes | Algorithm 1: Differentially Private SGD [1]... Algorithm 2: Finding Sample Parameters... Algorithm 3: Finding Scale Parameters... Algorithm 4: Subroutine get Sample Rate... Algorithm 5: Subroutine get Noise. |
| Open Source Code | No | The paper states 'we implement our methods in Python 3.9 and extend standard Opacus' and provides a link to Opacus. However, it does not provide concrete access to the source code for their own individualized privacy methods (IDP-SGD) or explicitly state that their code is open-sourced or available. |
| Open Datasets | Yes | We perform evaluation on the MNIST [23], SVHN [29], and CIFAR10 [22] dataset |
| Dataset Splits | No | To evaluate the utility of our methods, we use the datasets standard train-test splits and report test accuracies. This statement only explicitly mentions "train-test splits" and does not provide specific details on a separate validation set split or how it's handled to fully reproduce the experimental setup for all three (train/validation/test). |
| Hardware Specification | Yes | we required, in total less than 16h of GPU time on a standard Ge Force RTX 2080 Ti... We ran the experiment on combining individualized privacy assignment and accounting on the same machines RTX 2080Ti and the total compute time is also around 2h. To train all the shadow models for our membership inference attack and run inference on them, we ran on an A100 GPU and required a total runtime of roughly 32 hours. |
| Software Dependencies | Yes | For our empirical evaluation, we implement our methods in Python 3.9 and extend standard Opacus with our individualized privacy parameters and a per-privacy group accounting. (Implied version for Opacus via git commit hash in URL [32] 211acbab1734c865dcfee9edd106c954c58cb10c for the referenced function get_noise_multiplier.) |
| Experiment Setup | Yes | The training and standard DP-SGD and IDP-SGD hyperparameters are specified in Table 5 and Table 6 in Appendix D, respectively where the noise multiplier σ is derived with the function get_noise_multiplier provided in Opacus [32]. It takes in as arguments the specified parameters δ, q, I, and target ε. |