Robustness to Programmable String Transformations via Augmented Abstract Training
Authors: Yuhao Zhang, Aws Albarghouthi, Loris D’Antoni
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we evaluate A3T by answering the following research questions: RQ1: Does A3T improve robustness in rich perturbation spaces for character-level and word-level models? RQ2: How does the complexity of the perturbation space affect the effectiveness of A3T? and We show the results for the selected perturbation spaces on character-level and word-level models in Tables 3, 4, and 5, respectively. |
| Researcher Affiliation | Academia | Yuhao Zhang 1 Aws Albarghouthi 1 Loris D Antoni 1 1Department of Computer Science, University of Wisconsin Madison, Madison, USA. |
| Pseudocode | Yes | Algorithm 1 A3T |
| Open Source Code | Yes | 1We provide our code at https://github.com/Forev er Zyh/A3T. |
| Open Datasets | Yes | AG News (Zhang et al., 2015) dataset consists of a corpus of news articles collected by Gulli (2005) and SST2 (Socher et al., 2013) is the Stanford Sentiment Treebank dataset |
| Dataset Splits | Yes | We split the first 4,000 training examples for validation. and The dataset contains 67,349 training, 872 validation, and 1,821 testing examples for each class. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, processor types, memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'Tensor Flow' and 'Keras' in links related to datasets, but does not provide specific version numbers for these or any other software dependencies needed to replicate the experiment. |
| Experiment Setup | Yes | In all augmentation training baselines, and A3T, we also adopt a curriculum-based training method (Huang et al., 2019; Gowal et al., 2019) which uses a hyperparameter λ to weigh between normal loss and maximization objective in Eq. (2). |