Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Differentially Private Sharpness-Aware Training
Authors: Jinseong Park, Hoki Kim, Yujin Choi, Jaewook Lee
ICML 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experimental results demonstrate that the proposed method improves the performance of deep learning models with DP from both scratch and finetuning. Code is available at https://github. com/jinseong P/DPSAT. |
| Researcher Affiliation | Academia | 1Department of Industrial Engineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 08826, South Korea. 2Institute of Engineering Research, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 08826, South Korea. |
| Pseudocode | Yes | Algorithm 1 DP-SAT Input: Initial parameter w0, learning rate η, radius ρ, clipping threshold C, variance σ2 from Proposition 2.2, and small τ to prevent zero division. Output: Final parameter w T . |
| Open Source Code | Yes | Code is available at https://github. com/jinseong P/DPSAT. |
| Open Datasets | Yes | For the empirical results trained from scratch, we evaluate the performance of our method on three commonly used benchmarks for differentially private deep learning: MNIST, Fashion MNIST, CIFAR-10, and SVHN. |
| Dataset Splits | Yes | The training data for each dataset was partitioned into training and test sets with a ratio of 0.8:0.2, and the test accuracy was averaged over 5 different random seeds for each dataset. |
| Hardware Specification | Yes | All experiments are conducted using the Py Torch-based libraries (Kim, 2020; Yousefpour et al., 2021) with Python on four NVIDIA Ge Force RTX 3090 GPUs. |
| Software Dependencies | No | The paper mentions 'Py Torch-based libraries' and 'Python' but does not provide specific version numbers for any software dependencies, which is required for reproducibility. |
| Experiment Setup | Yes | We use SGD as a base optimizer with a momentum of 0.9 and a learning rate of 2.0, without any learning rate decay, as mentioned in (Cheng et al., 2022). We conducted a hyperparameter search on ρ = {0.005, 0.01, 0.02, 0.03, 0.05, 0.1}, and the privacy broken probability δ = 10 5 in DP training. Table 5. Hyperparameters for training on MNIST, Fashion MNIST, CIFAR-10, and SVHN. |