Evolution of Neural Tangent Kernels under Benign and Adversarial Training
Authors: Noel Loo, Ramin Hasani, Alexander Amini, Daniela Rus
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we perform an empirical study of the evolution of the empirical NTK under standard and adversarial training, aiming to disambiguate the effect of adversarial training on kernel learning and lazy training. |
| Researcher Affiliation | Academia | Noel Loo, Ramin Hasani, Alexander Amini, Daniela Rus Computer Science and Artificial Intelligence Lab (CSAIL) Massachusetts Institute of Technology (MIT) {loo, rhasani, amini, rus} @mit.edu |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at https://github.com/yolky/adversarial_ntk_evolution |
| Open Datasets | Yes | We train Resnet18s on CIFAR-10 or CIFAR-100 [41] |
| Dataset Splits | No | The paper mentions using CIFAR-10 or CIFAR-100 but does not explicitly provide specific details on the train/validation/test dataset splits, such as percentages or sample counts. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware used for running its experiments, such as GPU models, CPU types, or cloud compute instances. While the authors' checklist states 'Yes' to including this information, it is not present in the provided paper text. |
| Software Dependencies | No | The paper mentions using the JAX library and Haiku, citing their publication years (2018 and 2020, respectively) but does not provide specific version numbers for these software dependencies, such as 'JAX vX.Y.Z'. |
| Experiment Setup | Yes | We train Resnet18s on CIFAR-10 or CIFAR-100 [41] for t epochs either using benign data (i.e. no data modification), or adversarial training, for 0 t 100. ... For all experiments in the main text, we use the standard ε = 4/255 adversarial radius under a L norm, but we verify that the results hold for ε = 8/255 in section 9 with additional results for ε = 8/255 in the appendix. ... We use learning rates of either η = 0.0001 or η = 0.01 |