Robustness Verification for Transformers
Authors: Zhouxing Shi, Huan Zhang, Kai-Wei Chang, Minlie Huang, Cho-Jui Hsieh
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4 EXPERIMENTS To demonstrate the effectiveness of our algorithm, we compute certified bounds for several sentiment classification models and perform an ablation study to show the advantage of combining the backward and forward processes. We also demonstrate the meaningfulness of our certified bounds with an application on identifying important words. |
| Researcher Affiliation | Academia | Zhouxing Shi1, Huan Zhang2, Kai-Wei Chang2, Minlie Huang1, Cho-Jui Hsieh2 1Dept. of Computer Science & Technology, Tsinghua University, Beijing 10084, China 2Dept. of Computer Science, University of California, Los Angeles, CA 90095, USA |
| Pseudocode | No | The paper provides mathematical derivations and descriptions of the method but does not include a structured pseudocode or algorithm block. |
| Open Source Code | No | The paper does not include an explicit statement about releasing source code or a link to a code repository. |
| Open Datasets | Yes | We use two datasets: Yelp (Zhang et al., 2015) and SST (Socher et al., 2013). |
| Dataset Splits | Yes | Yelp consists of 560,000/38,000 examples in the training/test set and SST consists of 67,349/872/1,821 examples in the training/development/test set. |
| Hardware Specification | Yes | Experiments are conducted on an NVIDIA TITAN X GPU. |
| Software Dependencies | No | The paper does not specify versions for any software dependencies or libraries used for implementation or experiments. |
| Experiment Setup | Yes | For the main experiments, we consider N-layer models (N 3), with 4 attention heads, hidden sizes of 256 and 512 for self-attention and feed-forward layers respectively, and we use Re LU activations for feed-forward layers. |