Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching
Authors: Jonas Geiping, Liam H Fowl, W. Ronny Huang, Wojciech Czaja, Gavin Taylor, Michael Moeller, Tom Goldstein
ICLR 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct an experimental evaluation, showing that poisoned datasets created by this method are robustly compromised and significantly outperform other attacks on CIFAR-10 on the benchmark of Schwarzschild et al. (2020). We then demonstrate reliably successful attacks on common Image Net models in realistic training scenarios. |
| Researcher Affiliation | Academia | Jonas Geiping Dep. of Electr. Eng. and Computer Science University of Siegen EMAIL Liam Fowl Department of Mathematics University of Maryland EMAIL W. Ronny Huang Department of Computer Science University of Maryland EMAIL Wojciech Czaja Department of Mathematics University of Maryland EMAIL Gavin Taylor Computer Science US Naval Academy EMAIL Michael Moeller Dep. of Elect. Eng. and Computer Science University of Siegen EMAIL Tom Goldstein Department of Computer Science University of Maryland EMAIL |
| Pseudocode | Yes | Algorithm 1 Poison Brewing via the discussed approach. |
| Open Source Code | Yes | Code for all experiments can be found at https://github.com/Jonas Geiping/ poisoning-gradient-matching. |
| Open Datasets | Yes | We conduct an experimental evaluation, showing that poisoned datasets created by this method are robustly compromised and significantly outperform other attacks on CIFAR-10 on the benchmark of Schwarzschild et al. (2020). We then demonstrate reliably successful attacks on common Image Net models in realistic training scenarios. |
| Dataset Splits | Yes | We apply algorithm 1 with the following hyperparameters for all our experiments: τ = 0.1, R = 8, M = 250. We train victim models in a realistic setting, considering data augmentation, SGD with momentum, weight decay and learning rate drops. ... For CIFAR-10 we add data augmentations using horizontal flipping with probability 0.5 and random crops of size 32 32 with zero-padding of 4. For Image Net we resize all images to 256 256 and crop to the central 224 224 pixels. We also consider horizontal flipping with probability 0.5, and data augmentation with random crops of size 224 224 with zero-padding of 28. |
| Hardware Specification | Yes | We use a heterogeneous mixture of hardware for our experiments. CIFAR-10, and a majority of the Image Net experiments, were run on NVIDIA GEFORCE RTX 2080 Ti gpus. CIFAR-10 experiments were run on 1 gpu, while Image Net experiments were run on 4 gpus. We also use NVIDIA Tesla P100 gpus for some Image Net experiments. All timed experiments were run using 2080 Ti gpus. |
| Software Dependencies | No | No specific version numbers for software dependencies were found. For example, 'We train the Conv Net, Mobile Net-v2 and VGG-16 with initial learning rate of 0.01...' mentions software but not specific versions. |
| Experiment Setup | Yes | We train the Conv Net, Mobile Net-v2 and VGG-16 with initial learning rate of 0.01 and the residual architectures with initial learning rate 0.1. We train for 40 epochs, dropping the learning rate by a factor of 10 at epochs 14, 24, 35. We train with stochastic mini-batch gradient descent with Nesterov momentum, with batch size 128 and momentum 0.9. Note that the dataset is shuffled in each epoch, so that where poisoned images appear in mini-batches is random and not known to the attacker. We add weight decay with parameter 5 10 4. For CIFAR-10 we add data augmentations using horizontal flipping with probability 0.5 and random crops of size 32 32 with zero-padding of 4. For Image Net we resize all images to 256 256 and crop to the central 224 224 pixels. We also consider horizontal flipping with probability 0.5, and data augmentation with random crops of size 224 224 with zero-padding of 28. |