Generalised Lipschitz Regularisation Equals Distributional Robustness
Authors: Zac Cranko, Zhan Shi, Xinhua Zhang, Richard Nock, Simon Kornblith
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | As our experiments show, this method achieves higher robustness than state of the art (Cisse et al., 2017; Anil et al., 2019). We studied the empirical robustness and accuracy of the proposed Lipschitz regularisation technique for adversarial training of kernel methods, under both Gaussian kernel and inverse kernel. |
| Researcher Affiliation | Collaboration | 1Universität Tübingen, Tübingen, Germany 2University of Illinois at Chicago, IL, USA 3Google Brain. |
| Pseudocode | Yes | Algorithm 1 Training L-Lipschitz binary SVM |
| Open Source Code | No | The paper references 'LNets. https://github.com/cemanil/LNets.' in the bibliography, which is a third-party tool used for comparison. However, the authors do not state that they are releasing their own source code for the methodology described in this paper. |
| Open Datasets | Yes | Datasets We tested on three datasets: MNIST, Fashion-MNIST, and CIFAR10. |
| Dataset Splits | Yes | The number of training/validation/test examples for the three datasets are 54k/6k/10k, 54k/6k/10k, 45k/5k/10k, respectively. |
| Hardware Specification | No | The paper does not specify the exact hardware (e.g., GPU models, CPU types, memory) used for running the experiments. It only implies computations were performed. |
| Software Dependencies | No | The paper mentions using PGD and L-BFGS, but it does not specify any software names with version numbers (e.g., specific library versions for PyTorch, TensorFlow, or scikit-learn). |
| Experiment Setup | Yes | The perturbation δ was constrained in an 2-norm or 1-norm ball. To evaluate robustness, we scaled the perturbation bound δ from 0.1 to 0.6 for 1-norm norm, and from 1 to 6 for 2-norm norm (when δ = 6, the average magnitude per coordinate is 0.214). We normalised gradient and fine-tuned the step size. To defend against 2-norm attacks, we set L = 100 for all algorithms. Gauss-Lip achieved high accuracy and robustness on the validation set with bandwidth σ = 1.5 for Fashion MNIST and CIFAR-10, and σ = 2 for MNIST. To defend against 1-norm attacks, we set L = 1000 for all the four methods as in Anil et al. (2019). The best σ for Gauss-Lip is 1 for all datasets. Inverse-Lip used 5 layers. |