Practical Differentially Private Hyperparameter Tuning with Subsampling
Authors: Antti Koskela, Tejas D. Kulkarni
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We carry out experiments on several standard datasets, where we are able to improve upon the baseline tuning method by a clear margin. |
| Researcher Affiliation | Industry | Antti Koskela Nokia Bell Labs antti.h.koskela@nokia-bell-labs.com Tejas Kulkarni Nokia Bell Labs tejas.kulkarni@nokia-bell-labs.com |
| Pseudocode | No | The paper describes its method in numbered steps (e.g., 'Our method works as below:'), but does not provide a formally labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | No | The paper states 'For the implementation of DP-SGD and DP-Adam, we use the Opacus library (Yousefpour et al., 2021).' This refers to a third-party library used, not the open-source release of the specific methodology presented in this paper. |
| Open Datasets | Yes | We perform our evaluations on standard benchmark datasets for classification: CIFAR-10 (Krizhevsky and Hinton, 2009), MNIST (Le Cun et al., 1998), Fashion MNIST (Xiao et al., 2017) and IMDB (Maas et al., 2011). |
| Dataset Splits | No | In all of our experiments, we have a partitioning of the available data into train and test sets and we choose the best model based on the test accuracy. [...] the approach of making only two (train and test) partitions of available data instead of three (train, validation, and test) has been considered in many prior works. |
| Hardware Specification | No | The paper mentions 'a dedicated multi-GPU cluster' and 'two sets of gpus' but does not specify exact GPU models (e.g., NVIDIA A100) or other detailed hardware specifications like CPU models or memory. |
| Software Dependencies | No | The paper mentions using the 'Opacus library (Yousefpour et al., 2021)' and 'Ray Tune (Liaw et al., 2018)' but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | Yes | Full details of the experiments are given in Appendix A. [...] The learning rate grid size is either 9 or 10 and the grid values are specified in Table 2 (Appendix). We fix the subsampling ratio γ and the number of epochs to the values given in Table 1 (Appendix) for all models. [...] The number of trainable parameters and the hyperparameter grids are provided in Table 2 (Appendix B). |