Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Rethinking Backdoor Attacks
Authors: Alaa Khaddaj, Guillaume Leclerc, Aleksandar Makelov, Kristian Georgiev, Hadi Salman, Andrew Ilyas, Aleksander Madry
ICML 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically verify the efficacy of this algorithm on a variety of standard backdoor attacks. and Overall, our contributions are as follows: ... We show how to detect backdoor attacks under the corresponding assumption (i.e., that the backdoor trigger is the strongest feature in the dataset). We provide theoretical guarantees on our approach s effectiveness at identifying backdoored inputs, and demonstrate experimentally that our resulting algorithm is effective in a range of settings. |
| Researcher Affiliation | Academia | 1MIT. Correspondence to: Alaa Khaddaj <EMAIL>. |
| Pseudocode | No | The paper describes the steps of its algorithm, such as the local search algorithm in Section 4.2, in paragraph form but does not provide structured pseudocode or a formally labeled algorithm block. |
| Open Source Code | No | Our implementation and configuration files will be available in our code. (This indicates future availability, not current concrete access.) |
| Open Datasets | Yes | For all of these experiments, we use the CIFAR-10 dataset (Krizhevsky, 2009) |
| Dataset Splits | Yes | Specifically, for each experiment and setup, we train a total of 100,000 models, each on a random subset containing 50%6 of CIFAR-107, and chosen uniformly at random. and we train a model on the backdoored dataset, and compute the accuracy of this model on (a) the clean validation set, (b) and on the backdoored validation set8. |
| Hardware Specification | Yes | The speedup from using FFCV allows us to train a model to convergence in 40 seconds, and 100k models for each experiment using 16 V100 in roughly 1 day13. |
| Software Dependencies | No | The paper mentions software like the 'FFCV library' and 'Gurobi', and refers to a ResNet-9 architecture implementation, but it does not provide specific version numbers for these or other key software dependencies required for reproducibility. |
| Experiment Setup | Yes | We show the hyperparameter details in Table 512. Table 5: Hyperparameters used to train Res Net-9 on CIFAR10. Optimizer SGD, Epochs 24, Batch Size 1,024, Peak LR 0.05, Cyclic LR, Peak Epoch, Momentum 0.9, Weight Decay 4e-5. |