Beyond Uniform Lipschitz Condition in Differentially Private Optimization
Authors: Rudrajit Das, Satyen Kale, Zheng Xu, Tong Zhang, Sujay Sanghavi
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We verify the efficacy of our recommendation via experiments on 8 datasets. In Section 5.2, we corroborate our recommendation with experiments on four1 vision datasets, viz., Caltech-256 (Griffin et al., 2007), Food-101 (Bossard et al., 2014), CIFAR-100 and CIFAR-10, and two language datasets, viz., Tweet Eval Emoji (Barbieri et al., 2018) and Emotion (Saravia et al., 2018). |
| Researcher Affiliation | Collaboration | Rudrajit Das * 1 Satyen Kale 2 Zheng Xu 2 Tong Zhang 2 3 Sujay Sanghavi 1 1UT Austin 2Google Research 3HKUST. |
| Pseudocode | Yes | Algorithm 1 DP-SGD (Abadi et al., 2016) |
| Open Source Code | No | The paper mentions using a third-party library ('Py Torch s Opacus library') but does not state that the authors' own implementation code for the described methodology is open-source or provide a link to it. |
| Open Datasets | Yes | Our experiments here are conducted on four vision datasets available in Torchvision, viz., Caltech-256 (Griffin et al., 2007), Food-101 (Bossard et al., 2014), CIFAR-100 and CIFAR-10, and two language datasets, viz., Tweet Eval Emoji (Barbieri et al., 2018) and Emotion (Saravia et al., 2018). |
| Dataset Splits | No | The paper refers to 'test accuracy' and hyperparameter tuning but does not specify the exact train/validation/test split percentages, sample counts, or methodology used for data partitioning. |
| Hardware Specification | Yes | A single NVIDIA TITAN Xp GPU was used for all the experiments in this paper. |
| Software Dependencies | No | Py Torch s Opacus library (Yousefpour et al., 2021) is used for private training. (No version numbers provided for PyTorch or Opacus). |
| Experiment Setup | Yes | We consider three privacy levels (2, 10 5)-DP, (4, 10 5)-DP and (6, 10 5)-DP, with batch size = 500. We test several values of the clip norm τ, viz., the 0th, 10th, 20th, 40th, 80th and 100th percentile of the per-sample Lipschitz constants... For each value of τ, we tune over several values of the constant learning rate η, viz., {0.0001, 0.0003, 0.0006, 0.001, 0.003, 0.006, 0.01, 0.03, 0.06, 0.1, 0.3, 0.6, 1, 3, 6, 10}. |