LabelDP-Pro: Learning with Label Differential Privacy via Projections
Authors: Badih Ghazi, Yangsibo Huang, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Chiyuan Zhang
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that such projection-based algorithms can be made practical and that they improve on the state-of-the-art for label DP training in the high-privacy regime. We complement our empirical evaluation with theoretical results shedding light on the efficacy of our method through the lens of bias-variance trade-offs. |
| Researcher Affiliation | Collaboration | Badih Ghazi Google Research Yangsibo Huang Princeton University Pritish Kamath Google Research Ravi Kumar Google Research Pasin Manurangsi Google Research Chiyuan Zhang Google Research |
| Pseudocode | Yes | Algorithm 1: Label DP-Pro |
| Open Source Code | No | The paper does not provide a specific link or explicit statement about releasing the source code for the methodology described. |
| Open Datasets | Yes | We compare our Label DP-Pro on four benchmark datasets commonly used in Label DP learning evaluations: MNIST (Le Cun et al., 1998), k MNIST (Clanuwat et al., 2018), Fashion MNIST (Xiao et al., 2017), and CIFAR-10 (Krizhevsky et al., 2009). |
| Dataset Splits | No | The paper does not provide explicit train/validation/test split percentages or absolute counts for all datasets. For LP-2ST baseline, it mentions: 'We use 40% of the data for the first stage and 60% for the second.' but this is not a general validation split. |
| Hardware Specification | Yes | Results for MNIST, CIFAR-10, and CIFAR10-Self SL are reported on a single NVIDIA Tesla-P100 GPU, while results for Criteo Attribution are reported on a single NVIDIA A100 GPU. |
| Software Dependencies | No | The paper mentions 'all implemented in JAX' and refers to PyTorch and TensorFlow for autodiff primitives, but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | Yes | Table 8: Hyper-parameters for our experimental results presented in the main paper. Optimizer: SGD, RMSprop; Learning rate: various values; # Epochs: various values; Batch size: various values; Clipping norm (DP-SGD & Label DP-Pro): various values. |