What Can the Neural Tangent Kernel Tell Us About Adversarial Robustness?
Authors: Nikolaos Tsilivis, Julia Kempe
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments confirm a very strong alignment of loss gradients from the neural nets and the NTK across the whole duration of training, as can be seen in Fig. 3 (top). Then, as expected, kernel-generated attacks produce a similar drop in accuracy throughout training as the networks own white-box attacks, eventually driving robust accuracy to 0%, as seen in Fig. 3 (bottom). |
| Researcher Affiliation | Academia | Nikolaos Tsilivis Center for Data Science New York University nt2231@nyu.edu Julia Kempe Center for Data Science and Courant Institute of Mathematical Sciences New York University kempe@nyu.edu |
| Pseudocode | No | No pseudocode or algorithm blocks are present in the paper. |
| Open Source Code | Yes | We provide code to visualize features induced by kernels, giving a unique and principled way to inspect features induced by standardly trained nets (available at https://github.com/ Tsili42/adv-ntk). |
| Open Datasets | Yes | To this end, we consider classification problems from MNIST (10 classes) and CIFAR-10 (car vs airplane). We compose the Gram matrices from the whole training dataset (50000 and 10000, respectively)... |
| Dataset Splits | No | The paper mentions using 'a hold-out validation set' but does not provide specific details on the dataset splits (percentages, sample counts, or citations to predefined splits). |
| Hardware Specification | No | The paper states support from 'NYU IT High Performance Computing resources', but does not specify any particular GPU models, CPU models, or other detailed hardware specifications used for the experiments. |
| Software Dependencies | No | The paper mentions the use of 'Neural Tangents library' and 'JAX' but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | No | The paper mentions 'small learning rate' and 'PGD with an ℓ∞ constraint' for training but defers 'Full implementations details' to Appendix E, which is not provided, thus lacking specific hyperparameter values in the main text. |