Differentially Private Learning with Per-Sample Adaptive Clipping
Authors: Tianyu Xia, Shuheng Shen, Su Yao, Xinyi Fu, Ke Xu, Xiaolong Xu, Xing Fu
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In addition, through extensive experimental evaluation, we show that DP-PSAC outperforms or matches the state-of-the-art methods on multiple main-stream vision and language tasks. |
| Researcher Affiliation | Collaboration | 1Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University 2Department of Computer Science & Technology, Tsinghua University 3Zhongguancun Laboratory, Beijing 4School of Software & Microelectronics, Peking University 5Tiansuan Lab, Ant Group |
| Pseudocode | Yes | Algorithm 1: DP-PSAC Input: initial weights x0 ,learning rate ηt , batch size B, dataset S = (z1, ..., z N), privacy budget (ϵ, δ), max clipping threshold C, the number of iterations T |
| Open Source Code | No | The paper discusses software libraries used (PyTorch, Opacus, private-transformers, transformers) but does not provide an explicit statement or link for the open-source code of their proposed DP-PSAC method. |
| Open Datasets | Yes | We conduct extensive experiments on multiple image classification datasets, including MNIST (Le Cun et al. 1998), Fashion MNIST (Xiao, Rasul, and Vollgraf 2017), CIFAR10 (Krizhevsky, Hinton et al. 2009), imagenette (a subset of imagenet (Deng et al. 2009) with ten labels), and Celeb A (Liu et al. 2015). We used four sentence classification datasets from the GLUE benchmark dataset, including MNLI (multi-genre inference) (Williams, Nangia, and Bowman 2017), QQP (equivalence classification), QNLI (Questionanswering inference) (Rajpurkar et al. 2016), and SST-2 (sentiment classification) (Socher et al. 2013). |
| Dataset Splits | No | The paper states that for MNIST and Fashion MNIST, they used 'the same settings as Tramer and Boneh (2020)', and for CIFAR10, 'the same experimental setup as Tramer and Boneh (2020)'. For other datasets, the setup is 'the same as previous works (Klause et al. 2022; Bu et al. 2022)'. The paper does not explicitly detail the train/validation/test splits within its own text. |
| Hardware Specification | Yes | All experiments are performed on a server with an Intel Xeon Platinum 8369B CPU, an NVIDIA A100 GPU, and 125GB memory. |
| Software Dependencies | Yes | The natural language processing experiments are based on private-transformers (Li et al. 2021) of version 0.1.0, transformers of version 4.11.3, and the latest version of Pytorch. |
| Experiment Setup | No | The paper generally refers to external works for experimental settings, stating 'we adopt the same settings as Bu et al. (2022)' for Auto-S/NSGD, and 'have the same settings as Tramer and Boneh (2020)' for CNN models on MNIST/Fashion MNIST. Specific hyperparameter values (e.g., learning rate, batch size, epochs) used for the main reported results are not explicitly listed within the text. |