Knowledge Enhanced Machine Learning Pipeline against Diverse Adversarial Attacks
Authors: Nezihe Merve Gürel, Xiangyu Qi, Luka Rimanic, Ce Zhang, Bo Li
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5. Experimental Evaluation In this section, we evaluate KEMLP based on the traffic sign recognition task against different adversarial attacks and corruptions, including the physical attacks (Eykholt et al., 2018), L1 bounded attacks, unforeseen attacks (Kang et al., 2019), and common corruptions (Hendrycks and Dietterich, 2019). We show that under both whitebox and blackbox settings against a diverse set of attacks, 1) KEMLP achieves significant robustness gain over baselines, 2) KEMLP maintains similar clean accuracy with a strong main task model whose clean accuracy is originally high (e.g., vanillar CNN), 3) KEMLP even achieves higher clean accuracy than a relatively weak main task model whose clean accuracy is originally low as a tradeoff for its robustness (e.g., adversarially trained models). |
| Researcher Affiliation | Academia | 1ETH Zurich, Zurich, Switzerland 2Zhejiang University, China (work done during remote internship at UIUC) 3University of Illinois at Urbana-Champaign, Illinois, USA. |
| Pseudocode | No | No structured pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | Yes | Our code is publicly available for reputability 1. 1https://github.com/AI-secure/Knowledge-Enhanced-Machine-Learning-Pipeline |
| Open Datasets | Yes | Following existing work (Eykholt et al., 2018; Wu et al., 2019) that evaluate ML robustness on traffic sign data, we adopt LISA (Mogelmose et al., 2012) and GTSRB (Stallkamp et al., 2012) for training and evaluation. |
| Dataset Splits | Yes | In total, there are 14880 samples in the training set, 972 samples in the validation set, and 3888 samples in the test set |
| Hardware Specification | No | No specific hardware details (such as GPU or CPU models, memory specifications, or cloud instance types) used for running experiments were provided in the paper. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | More details on implementation are provided in Appendix B.3. ... We set β = 0.5 for KEMLP indicating a balanced random guess for the distribution tradeoff. |