The Dark Side of AutoML: Towards Architectural Backdoor Search
Authors: Ren Pang, Changjiang Li, Zhaohan Xi, Shouling Ji, Ting Wang
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | With extensive evaluation on benchmark datasets, we show that EVAS features high evasiveness, transferability, and robustness, thereby expanding the adversary s design spectrum. We conduct an empirical evaluation of EVAS on benchmark datasets under various scenarios. |
| Researcher Affiliation | Academia | Ren Pang Changjiang Li Zhaohan Xi Shouling Ji Ting Wang Pennsylvania State University, {rbp5354, cbl5583, zxx5113, ting}@psu.edu Zhejiang University, sji@zju.edu.cn |
| Pseudocode | Yes | Algorithm 1: EVAS Attack Input: n pool size; m sample size; score score function; sample subset sampling function; mutate arch mutation function; Output: exploitable arch |
| Open Source Code | Yes | The code is available at https://github.com/ain-soph/nas_backdoor. |
| Open Datasets | Yes | Datasets. In the evaluation, we primarily use three datasets that have been widely used to benchmark NAS methods (Chen et al., 2019; Li et al., 2020; Liu et al., 2019; Pham et al., 2018; Xie et al., 2019): CIFAR10 (Krizhevsky & Hinton, 2009), which consists of 32 32 color images drawn from 10 classes; CIFAR100, which is similar to CIFAR10 but includes 100 finer-grained classes; and Image Net16, which is a subset of the Image Net dataset (Deng et al., 2009) down-sampled to images of size 16 16 in 120 classes. |
| Dataset Splits | No | The paper does not explicitly provide training/test/validation dataset splits with specific percentages, sample counts, or references to predefined validation splits. While it mentions training models and evaluating performance, it does not detail a separate validation split for hyperparameter tuning or early stopping. |
| Hardware Specification | No | The paper does not specify any particular hardware used for running the experiments, such as GPU models, CPU types, or memory configurations. |
| Software Dependencies | No | The paper mentions optimizers like 'Adam' and 'SGD' but does not provide specific version numbers for any software dependencies or libraries required for replication. |
| Experiment Setup | Yes | Table 6. Default parameter setting. Table 7 lists the architecture of the trigger generator. |