Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Deep Learning with Plausible Deniability
Authors: Wenxuan Bao, Shan Jin, Hadi Abdullah, Anderson Nascimento, Vincent Bindschaedler, Yiwei Cai
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experiments 5.1 Experimental Setup 5.2 Evaluations Our findings, summarized in Table 1, show that PD-SGD achieves a superior privacy-utility trade-off, surpassing both empirical and DP-based defenses across all evaluated tasks. |
| Researcher Affiliation | Collaboration | Wenxuan Bao1 Shan Jin2 Hadi Abdullah2 Anderson C. A. Nascimento2 Vincent Bindschaedler1 Yiwei Cai2 1University of Florida 2Visa Research |
| Pseudocode | Yes | Algorithm 1 Plausibly Deniable Stochastic Gradient Descent (PD-SGD) Algorithm 2 Hyperparameter Search |
| Open Source Code | Yes | Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification:We provide anonymous github link for the code. |
| Open Datasets | Yes | We use three of the most commonly used datasets for evaluating membership inference attacks [38, 45, 40] and DP-SGD [12, 3]: CIFAR-10, CIFAR-100, and Purchase-100. For the models, we fine-tune Vi T-B-16 for CIFAR-10 and CIFAR-100, linear model for Purchase-100, and Wide Res Net for CIFAR-10 and CIFAR-100 training from scratch. Vi T-B-16 are pre-trained on the LAION-2B dataset [37]. SST-2 [39] |
| Dataset Splits | Yes | CIFAR-10 [26] contains 60,000 images with 10 classes. We use 50,000 as the full training set and 10,000 as the test set as most papers do. Each example has three RGB channels and size 32 × 32 pixels. For fine-tuning tasks, we only use 500 data samples for training and for training from scratch tasks, we use 30,000 for training. CIFAR-100 contains 60,000 color images, each with a resolution of 32 × 32 pixels. The dataset allocation includes 50,000 images for training purposes and 10,000 for testing. For the fine-tuning task, we only use 1000 data samples for training and the rest of training data examples are used for MIA evaluation. For training from scratch, we use 25,000 data samples as the same setting in [50]. Purchase-100...For training, we use 25,000 samples and the rest for testing. For MIAs, we use 25,000 samples from test set as shadow dataset. When evaluating attacks, we always use balanced evaluation dataset (50% member and 50% non-member). |
| Hardware Specification | Yes | We conducted additional experiments on an NVIDIA B200 GPU. Full details on computational time and memory usage are available in Appendix G.9. Table 15: GPU Memory Usage Comparison: Evaluation of GPU memory usage of PD-SGD against DP-SGD and non-private (SGD) training with the Wide-Res Net-16-4 model on CIFAR-10 dataset |
| Software Dependencies | No | We implemented PD-SGD using Py Torch. For DP-SGD, we use Opacus [47]. For other empirical defense mechanisms, we reproduce them using SELENA s [40] original code-base 6 and HAMP s original code-base 7. For membership inference attack, we use the Privacy Meter toolbox.8 |
| Experiment Setup | Yes | Unless otherwise stated, we instantiate the privacy test using simple counting without randomizing the threshold. We tune privacy parameters T, γ (= ln α), and σ according to Appendix E. In cases where we randomized the threshold and used a ceiling, we set β = e and ψ = 0.2. Table 4: Hyperparameters setting for experiments in Table 1 Dataset Param setting σ γ T Step Reject Rate CIFAR-10(FT) PS 1 0.1 40 2 20000 27.78% CIFAR-100(FT) PS 1 0.1 50 3 20000 44.08% |