Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
DPVIm: Differentially Private Variational Inference Improved
Authors: Joonas Jälkö, Lukas Prediger, Antti Honkela, Samuel Kaski
TMLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the efficacy of our proposed improvements through various experiments on real data. [...] We experimentally test our methods for two different tasks using mean-field approximation with real data: learning a probabilistic generative model for private data sharing and learning a logistic regression model. We also experimentally explore aligned DPVI with full-rank Gaussian approximation using simulated data. |
| Researcher Affiliation | Academia | Joonas Jälkö EMAIL Department of Computer Science, University of Helsinki Lukas Prediger EMAIL Department of Computer Science, Aalto University Antti Honkela EMAIL Department of Computer Science, University of Helsinki Samuel Kaski EMAIL Department of Computer Science, Aalto University Department of Computer Science, University of Manchester |
| Pseudocode | Yes | Algorithm 1 The aligned gradient procedure (single step) Algorithm 2 The aligned natural gradient procedure (single step) |
| Open Source Code | Yes | The code for reproducing the experiments can be found at https://github.com/DPBayes/dpvim-experiments. |
| Open Datasets | Yes | A recent study by Niedzwiedz et al. (2020) on personal health data from the United Kingdom Biobank (UKB) (Sudlow et al., 2015)... ...we further demonstrate our methods on the publicly available Adult data set from the UCI machine learning repository (Dua & Graff, 2017)... ...US Census 1990 data (from UCI Dua & Graff (2017)) |
| Dataset Splits | Yes | In this experiment we create simulated data where we control the amount of correlations between data dimensions as the ratio ρ of non-zero off-diagonal entries in the correlation matrix. To generate data with d dimensions and correlation density ρ, we ... sample N = 10 000 data points xn N(0, Σ) ... For the same d, ρ we then generate another 10 000 data points and compute the log-likelihood using the obtained posterior approximation. |
| Hardware Specification | Yes | All runs were executed on a computing cluster utilising Nvidia K80, A100, P100, V100 GPU hardware |
| Software Dependencies | No | The paper mentions 'd3p package (Prediger et al., 2022) for the Num Pyro probabilistic programming framework (Phan et al., 2019; Bingham et al., 2019)' and 'Adam (Kingma & Ba, 2015) as the optimiser'. While these are software components, specific version numbers for d3p, Num Pyro, or Python are not provided, which is required for a reproducible description of software dependencies. |
| Experiment Setup | Yes | We use Adam (Kingma & Ba, 2015) as the optimiser for all the experiments with starting learning rate of 10-3. In all of our experiments, the δ privacy parameter was set to 1/N where N denotes the size of the training data. For the UKB experiment In the experiments, we used various different training lengths (depicted e.g. in Figure 2). For all of our runs, we set the subsampling rate as 0.01. The clipping threshold C was set to C = 2.0 for the aligned and vanilla, 4.0 for the preconditioned variant and to 0.1 for the natural gradient based variants. The training was run for 4 000 epochs with subsampling ratio of 0.01, corresponding to total of 400 000 gradient steps. We chose the clipping thresholds for the gradient perturbation algorithm as the 97.5% upper quantile of the training data gradient norms at the non-private optima. |