Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Certified Robustness against Sparse Adversarial Perturbations via Data Localization
Authors: Ambar Pal, Rene Vidal, Jeremias Sulam
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Empirical Evaluation In this section, we will brieļ¬y describe existing methods for probabilistic ā0 certiļ¬cation, (Levine & Feizi, 2020b) and (Jia et al., 2022) as well as deterministic ā0 certiļ¬cation (Hammoudeh & Lowd, 2023), and then empirically compare our (deterministic) ā0 certiļ¬ed defense Box-NN to these approaches. |
| Researcher Affiliation | Academia | Ambar Pal EMAIL Department of Computer Science & Mathematical Institute for Data Science Johns Hopkins University Baltimore, MD 21218, USA; RenƩ Vidal EMAIL Department of Electrical and Systems Engineering & Center for Innovation in Data Engineering and Science University of Pennsylvania Philadelphia, PA 19104, USA; Jeremias Sulam EMAIL Department of Biomedical Engineering & Mathematical Institute for Data Science Johns Hopkins University Baltimore, MD 21218, USA |
| Pseudocode | No | The paper describes the Box-NN classifier and methods for its development and certification (Section 4), including formulas for calculating distances (Lemma 4.1) and robustness certificates (Theorem 4.2). However, it does not include any explicitly labeled 'Pseudocode' or 'Algorithm' block with structured, code-like steps. |
| Open Source Code | No | The paper does not contain any explicit statement about releasing source code, nor does it provide a link to a code repository or mention code in supplementary materials. The link provided 'https: // openreview. net/ forum? id= 17Ld3davz F' is for the OpenReview forum for the paper. |
| Open Datasets | Yes | We provide empirical evaluation on the MNIST and the Fashion-MNIST datasets, and demonstrate that Box-NN obtains state-of-the-art results in certiļ¬ed ā0 robustness. |
| Dataset Splits | No | For each of the methods described so far, we plot Cert Acc against Ļµ using the corresponding robust classiļ¬er g and the certiļ¬cate C over samples from the test set of the datasets mentioned. The paper does not explicitly provide details about the training, validation, or test dataset splits (e.g., percentages, sample counts, or specific splitting methodology). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or cloud instance types) used for running the experiments. |
| Software Dependencies | No | We ablate over a few choices of the gradient-based optimizer for our problem: (a) vanilla SGD with a learning rate of 0.02, (b) SGD with a learning rate of 0.02, a momentum of 0.9, and a weight decay of 0.0005, and (c) Adam with a learning rate of 0.001, and standard decay factors, in Fig. 8. The paper mentions optimizers but does not specify any software libraries or their version numbers. |
| Experiment Setup | Yes | We initialize Īø by using a set of boxes deļ¬ned from the data. This is done by ļ¬rst drawing a subset T of size M uniformly at random from the training data-points, and then initializing Īø with axis-aligned boxes centered at these data-points, as {(B(x 0.1, x + 0.1), y): (x, y) T}... We clip the certiļ¬cates to 50. ... We ablate over a few choices of the gradient-based optimizer for our problem: (a) vanilla SGD with a learning rate of 0.02, (b) SGD with a learning rate of 0.02, a momentum of 0.9, and a weight decay of 0.0005, and (c) Adam with a learning rate of 0.001, and standard decay factors. |