Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
DP-ImgSyn: Dataset Alignment for Obfuscated, Differentially Private Image Synthesis
Authors: Efstathia Soufleri, Deepak Ravikumar, Kaushik Roy
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments on various image classification datasets show that when using similar DP training mechanisms, our framework performs better than generative techniques (up to 20% in terms of image classification accuracy).1 |
| Researcher Affiliation | Academia | Efstathia Soufleri EMAIL Department of Electrical and Computer Engineering Purdue University Deepak Ravikumar EMAIL Department of Electrical and Computer Engineering Purdue University Kaushik Roy EMAIL Department of Electrical and Computer Engineering Purdue University |
| Pseudocode | Yes | Algorithm 1: DP Image Synthesis Input: DP-trained teacher model M; k number of optimization iterations; synthesis learning rate γsyn; batch statistics µ, σ for the private set; batch of public images x P Output: ˆx, one batch of aligned synthetic images |
| Open Source Code | Yes | 1The PyTorch implementation can be found at https://github.com/Efstathia-Soufleri/DP-Img Syn |
| Open Datasets | Yes | To evaluate our proposal, we use the same vision datasets as previous works; specifically, we use MNIST (Le Cun et al., 1998), Fashion MNIST (Xiao et al., 2017), CIFAR-10 (Krizhevsky et al., 2009), Image Nette (ima, 2018), Celeb A-Hair (Long et al., 2021; Liu et al., 2015), Celeb A-Gender (Long et al., 2021; Liu et al., 2015), Tiny Image Net (Li et al., 2015), Places365 (Zhou et al., 2017), LSUN (Yu et al., 2015), and Textures (Cimpoi et al., 2014). |
| Dataset Splits | Yes | Table 12: Dataset statistics, training, and test set sizes for the datasets used. Dataset Train Set Size Test Set Size Resolution Number of Classes MNIST 60,000 10,000 28x28 10 Fashion MNIST 60,000 10,000 28x28 10 CIFAR-10 50,000 10,000 32x32 10 Imagenette 10,000 5,000 224x224 10 Celeb A-Hair 162,770 19,962 64x64 3 Celeb A-Gender 162,770 19,962 64x64 2 Tiny Image Net 100,000 10,000 32x32 200 Places365 1,803,460 10,000 32x32 365 LSUN 9,895,373 303,304 64x64 10 Textures 5,640 1,880 224x224 47 |
| Hardware Specification | Yes | DP-Img Syn implementation uses the Pytorch (Paszke et al., 2019) framework, and the experiments were conducted on NVIDIA GeForce GTX 1080 Ti with 11 GB of memory with the Ubuntu operating system. |
| Software Dependencies | No | DP-Img Syn implementation uses the Pytorch (Paszke et al., 2019) framework, and the experiments were conducted on NVIDIA GeForce GTX 1080 Ti with 11 GB of memory with the Ubuntu operating system. |
| Experiment Setup | Yes | For the DP statistics capture for the teacher model (Section 4.1 from the main paper), we used the hyperparameters reported in Table 5. The ϵ denotes the privacy budget, ηtr is the number of training epochs, Ωtr is the batch size used for DP-SGD training, γtr is learning rate used for training, C denotes the maximum norm limit for the gradient vector g (g/max(1, ||g||2/C)), σ controls the amount of noise added to g (g + N(0, σ2C2I)), ηbn is the number of epochs used for capturing batch statistics, and Ωbn is the batch size used for capturing batch statistics in Table 5. For the DP Image Synthesis (described in Section 4.2, and Algorithm 1 in the main paper), we use the Adam optimizer (Kingma & Ba, 2014) with synthesis learning rate γsyn = 0.1, betas β1 = 0.5, β2 = 0.99. For MNIST and Fashion MNIST, we use a batch size of 80; for Celeb A-Hair and Celeb A-Gender, we use a batch size of 60. We use Stochastic Gradient Descent (SGD) optimizer (eon Bottou, 1998) with a learning rate η = 0.1, momentum 0.9, and weight decay 1e-4 for training a student model on the synthetic images. We use the multistep learning rate scheduler with γ = 0.1 and milestones at 120, 150, and 180 epochs. We train the models for 200 epochs with 256 as batch size. The temperature value used in our simulations is T = 100 for MNIST and Fashion MNIST and T = 10 for Celeb A-Hair and Celeb A-Gender. |