Understanding Robust Learning through the Lens of Representation Similarities
Authors: Christian Cianfarani, Arjun Nitin Bhagoji, Vikash Sehwag, Ben Zhao, Heather Zheng, Prateek Mittal
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments highlight hitherto unseen properties of robust representations that we posit underlie the behavioral differences of robust networks. We utilize a powerful set of tools known as representation similarity metrics, across three vision datasets, to obtain layer-wise comparisons between robust and non-robust DNNs with different training procedures, architectural parameters and adversarial constraints. |
| Researcher Affiliation | Academia | Christian Cianfarani Department of Computer Science University of Chicago Arjun Nitin Bhagoji Department of Computer Science University of Chicago Vikash Sehwag Department of ECE Princeton University Ben Zhao Department of Computer Science University of Chicago Haitao Zheng Department of Computer Science University of Chicago Prateek Mittal Department of ECE Princeton University |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | We hope the lessons from this paper and our accompanying website 3 with open-sourced code spur the development of robust learning-specific architectures and training methods. 3 https://robustrs.github.io/ |
| Open Datasets | Yes | Our key findings, using both benign and adversarial inputs as probes over CIFAR-10 [32] and the Imagenette and Imagewoof [25] subsets of the Imagenet [10] dataset, are: and Models and datasets. We consider three commonly used image datasets: CIFAR-10 [32], Image Nette [25], and Image Woof [25], where the latter two are subsets of Image Net [11], which we use due to the high cost of adversarial training on the full Image Net dataset. |
| Dataset Splits | No | Table 1 shows 'Train' and 'Test' accuracies, implying a train/test split. However, it does not explicitly provide specific dataset split percentages, sample counts, or a detailed splitting methodology for reproducibility, nor does it cite a predefined split. |
| Hardware Specification | Yes | All of our experiments were run on machines with either NVidia Titan RTX GPUs with 24GB of memory or RTX A4000 GPUs with 16GB of memory. |
| Software Dependencies | No | The paper mentions techniques and tools used (e.g., CKA), but does not provide specific software dependencies like programming language versions or library versions (e.g., Python 3.x, PyTorch 1.x) that are required for reproducibility. |
| Experiment Setup | Yes | Adversarial training. We follow standard convention with 1 perturbations and use = 8 255 for CIFAR-10 and = 4 255 for Image Net based datasets. We use 10-step projected gradient descent (PGD) attack during training and use 20 steps at test time to calculate adversarial representations. Representation similarity. ... In online CKA, we use a batch-size of 1024 and take 3 passes over the dataset to reduce any stochasticity in the output similarity score. |