PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining
Authors: Mishaal Kazmi, Hadrien Lautraite, Alireza Akbari, Qiaoyue Tang, Mauricio Soroco, Tao Wang, Sébastien Gambs, Mathias Lécuyer
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present PANORAMIA, a privacy leakage measurement framework for machine learning models that relies on membership inference attacks using generated data as non-members. By relying on generated non-member data, PANORAMIA eliminates the common dependency of privacy measurement tools on in-distribution non-member data. As a result, PANORAMIA does not modify the model, training data, or training process, and only requires access to a subset of the training data. We evaluate PANORAMIA on ML models for image and tabular data classification, as well as on large-scale language models. |
| Researcher Affiliation | Academia | Mishaal Kazmi University of British Columbia Hadrien Lautraite University du Québec à Montréal Alireza Akbari Simon Fraser University Qiaoyue Tang University of British Columbia Mauricio Soroco University of British Columbia Tao Wang Simon Fraser University Sébastien Gambs University du Québec à Montréal Mathias Lécuyer University of British Columbia |
| Pseudocode | Yes | Algorithm 1 PANORAMIA |
| Open Source Code | Yes | Code available here: https://github.com/ubc-systopia/panoramia-privacy-measurement. |
| Open Datasets | Yes | For image classification, we consider the CIFAR10 Krizhevsky (2009), and Celeb A Liu et al. (2015) datasets... on a subset of Wiki Text-103 dataset Merity et al. (2016) |
| Dataset Splits | Yes | We split Df in DG, Dtr in, Dte in, with |Dte in| = m (from Algorithm 1)... For CIFAR10 dataset, we use a 10, 000 out of 50, 000 images from the training data of the target model to train the generative model (Appendix C.1)... In the main results in Table 3, for both the baseline and PANORAMIA classifiers, the training and validation sets consist of 20000 and 2000 samples, respectively, with equal member non-member distribution (Appendix C.2). |
| Hardware Specification | Yes | The compute resources used for MIA and Baseline training were mainly running all experiments on cloud-hosted VMs, using 1 v100l GPU, 4 nodes per task, and 32G memory requested for each job on the cloud cluster. (Appendix C.1)... The compute resources used for training the target models are 4 V100-32gb GPUs, running on cloud-hosted VMs. (Appendix C.2) |
| Software Dependencies | No | The paper mentions software libraries and frameworks (e.g., Style GAN2, Pytorch, Opacus, MST method) but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | Our MIA is a loss-based attack, which uses an ML model taking as input both a datapoint x and the value of the loss of target model f on point x. Appendix C details the architectures used for the attack model for each data modality. (Section 5.1)... For Res Net101 CIFAR10-based classification models, we use a training batch size of 32; for Wide Res Net-28-10, we use batch size 128. (Appendix C.1)... Both models are trained using a learning rate of 0.0005 over 100 training epochs. (Appendix C.3) |