Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
On Achieving Optimal Adversarial Test Error
Authors: Justin D. Li, Matus Telgarsky
ICLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We first elucidate various fundamental properties of optimal adversarial predictors: the structure of optimal adversarial convex predictors in terms of optimal adversarial zero-one predictors, bounds relating the adversarial convex loss to the adversarial zero-one loss, and the fact that continuous predictors can get arbitrarily close to the optimal adversarial error for both convex and zero-one losses. Applying these results along with new Rademacher complexity bounds for adversarial training near initialization, we prove that for general data distributions and perturbation sets, adversarial training on shallow networks with early stopping and an idealized optimal adversary is able to achieve optimal adversarial test error. By contrast, prior theoretical work either considered specialized data distributions or only provided training error guarantees. |
| Researcher Affiliation | Academia | Justin D. Li & Matus Telgarsky University of Illinois, Urbana-Champaign EMAIL |
| Pseudocode | No | The paper describes algorithms and training steps in prose but does not provide structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described in this paper. It mentions running 'Rice et al. s code' for a figure, but does not offer its own implementation. |
| Open Datasets | No | The paper refers to 'training points (xk, yk)n k=1 are drawn from the distribution D' in its theoretical setting, and mentions 'CIFAR-10' in the context of prior work, but it does not provide concrete access information (link, DOI, repository, or formal citation) for a specific publicly available or open dataset used in its own research. |
| Dataset Splits | No | The paper discusses the potential use of a 'validation set' in Section 4.2 as a practical alternative for a theoretical assumption, but it does not specify any dataset splits or validation methodology for empirical experiments conducted in the paper. |
| Hardware Specification | No | The paper is theoretical and does not describe any experiments that would require specific hardware. Therefore, no hardware specifications are mentioned. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers) needed to replicate its theoretical analysis. |
| Experiment Setup | Yes | Our adversarial training will be as follows. To get the next iterate Wi+1 from Wi for i 0 we will use gradient descent with Wi+1 = Wi η b RA(Wi). ... In Corollary 4.2 we will show we can set parameters so that all error terms are arbitrarily small. ... Setting ρ = eΘ(ϵ), η = eΘ(1/ϵ), t = eΩ with n satisfying n = eΩ max(1, τm)R2 ϵ/ϵ2. |