Label-Only Membership Inference Attacks
Authors: Christopher A. Choquette-Choo, Florian Tramer, Nicholas Carlini, Nicolas Papernot
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that training with differential privacy or strong ℓ2 regularization are the only current defenses that meaningfully decrease leakage of private information, even for points that are outliers of the training distribution. |
| Researcher Affiliation | Collaboration | 1University of Toronto and Vector Institute 2Stanford University 3Google. Correspondence to: Christopher A. Choquette-Choo <choquette.christopher@gmail.com>. |
| Pseudocode | No | The paper describes its methods in prose, but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/cchoquette/ membership-inference. |
| Open Datasets | Yes | We evaluate our attacks on 8 datasets used by the canonical work of Shokri et al. (2016). These include 3 computer vision tasks3... and 4 non-computer-vision tasks4... 3MNIST, CIFAR-10, and CIFAR-100: https://www.tensorflow.org/api_docs/python/tf/keras/datasets 4Adult Dataset: http://archive.ics.uci.edu/ml/ datasets/Adult Texas-100, Purchase-100, and Locations datasets: https://github.com/privacytrustlab/datasets |
| Dataset Splits | No | The paper mentions training data, held-out data, and test accuracy but does not specify explicit train/validation/test dataset splits with percentages or counts, nor does it explicitly mention a validation set. |
| Hardware Specification | No | The paper does not provide specific details regarding the hardware (e.g., GPU models, CPU types, or cloud instance specifications) used for running the experiments. |
| Software Dependencies | No | The paper mentions 'TensorFlow' in a footnote related to datasets, but it does not specify version numbers for TensorFlow or any other software dependencies, libraries, or programming languages used. |
| Experiment Setup | Yes | We train target models with data augmentation similar to 3.3 and focus on translations as they are most common in computer vision. We use a simple pipeline where all translations of each image is evaluated in a training epoch. ... (non-random) weight decay of magnitude 0.0005. |