HQA-Attack: Toward High Quality Black-Box Hard-Label Adversarial Attack on Text
Authors: Han Liu, Zhi Xu, Xiaotong Zhang, Feng Zhang, Fenglong Ma, Hongyang Chen, Hong Yu, Xianchao Zhang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experimental results on five text classification datasets, three natural language inference datasets and two real-world APIs have shown that the proposed HQA-Attack method outperforms other strong baselines significantly. |
| Researcher Affiliation | Academia | Han Liu Dalian University of Technology Dalian, China liu.han.dut@gmail.com Zhi Xu Dalian University of Technology Dalian, China xu.zhi.dut@gmail.com Xiaotong Zhang Dalian University of Technology Dalian, China zxt.dut@hotmail.com Feng Zhang Peking University Beijing, China zfeng.maria@gmail.com Fenglong Ma The Pennsylvania State University Pennsylvania, USA fenglong@psu.edu Hongyang Chen Zhejiang Lab Hangzhou, China dr.h.chen@ieee.org Hong Yu Dalian University of Technology Dalian, China hongyu@dlut.edu.cn Xianchao Zhang Dalian University of Technology Dalian, China xczhang@dlut.edu.cn |
| Pseudocode | Yes | The detailed algorithm procedure of HQA-Attack is given in Appendix B. |
| Open Source Code | Yes | The source code and demo are publicly available at https://github.com/HQA-Attack/HQAAttack-demo. |
| Open Datasets | Yes | We conduct experiments on five public text classification datasets MR [28], AG s News [42], Yahoo [42], Yelp [42], IMDB [22], and three natural language inference datasets SNLI [3], MNLI [33], m MNLI [33]. |
| Dataset Splits | No | The paper mentions using '1000 test examples' for experiments, but does not explicitly provide details about training or validation splits, percentages, or cross-validation setup for the datasets used. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory amounts used for running the experiments. It only implies that experiments were run, but without specifying the machines. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers, such as programming languages, libraries, or frameworks (e.g., Python version, TensorFlow/PyTorch version). |
| Experiment Setup | Yes | For the hyperparameters, we consistently set r 5 and k 5 for all the datasets. The detailed parameter investigation is provided in Appendix G. In addition, during the optimization procedure, if we re-optimize the same adversarial example three times and no new better adversarial examples are generated, we randomly go back to the last three or four adversarial example. We do not re-optimize an adversarial example more than two times. |