Certified Defenses for Data Poisoning Attacks
Authors: Jacob Steinhardt, Pang Wei W. Koh, Percy S. Liang
NeurIPS 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we find that even under a simple defense, the MNIST-1-7 and Dogfish datasets are resilient to attack, while in contrast the IMDB sentiment dataset can be driven from 12% to 23% test error by adding only 3% poisoned data. |
| Researcher Affiliation | Academia | Jacob Steinhardt Stanford University jsteinha@stanford.edu Pang Wei Koh Stanford University pangwei@cs.stanford.edu Percy Liang Stanford University pliang@cs.stanford.edu |
| Pseudocode | Yes | Algorithm 1 Online learning algorithm for generating an upper bound and candidate attack. |
| Open Source Code | Yes | The code and data for replicating our experiments is available on Git Hub (http: //bit.ly/gt-datapois) and Codalab Worksheets (http://bit.ly/cl-datapois). |
| Open Datasets | Yes | For MNIST-1-7, following Biggio et al. (2012), we considered binary classification between the digits 1 and 7; this left us with n = 13007 training examples of dimension 784. For Dogfish, which is a binary classification task, we used the same Inception-v3 features as in Koh and Liang (2017)... We ran both the upper bound relaxation and the IQP solver on two text datasets, the Enron spam corpus (Metsis et al., 2006) and the IMDB sentiment corpus (Maas et al., 2011). |
| Dataset Splits | No | The paper provides training set sizes and mentions evaluation on a test set, but it does not specify explicit train/validation/test dataset splits (e.g., percentages, counts, or explicit validation set usage) needed to reproduce the data partitioning. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts) used for running its experiments. |
| Software Dependencies | No | We used a combination of CVXPY (Diamond and Boyd, 2016), YALMIP (Löfberg, 2004), Se Du Mi (Sturm, 1999), and Gurobi (Gurobi Optimization, Inc., 2016) to solve the optimization. While tools are mentioned, specific version numbers are not provided for them. |
| Experiment Setup | Yes | Here ry, sy are thresholds (e.g., chosen so that 30% of the data is removed). |