Revisiting Hilbert-Schmidt Information Bottleneck for Adversarial Robustness
Authors: Zifeng Wang, Tong Jian, Aria Masoomi, Stratis Ioannidis, Jennifer Dy
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments on multiple benchmark datasets and architectures demonstrate that incorporating an HSIC bottleneck regularizer attains competitive natural accuracy and improves adversarial robustness, both with and without adversarial examples during training. |
| Researcher Affiliation | Academia | Zifeng Wang Northeastern University zifengwang@ece.neu.edu Tong Jian Northeastern University jian@ece.neu.edu Aria Masoomi Northeastern University masoomi.a@northeastern.edu Stratis Ioannidis Northeastern University ioannidis@ece.neu.edu Jennifer Dy Northeastern University jdy@ece.neu.edu |
| Pseudocode | Yes | Algorithm 1: Robust Learning with HBa R Input: input sample tuples {(xi, yi)}n i=1, kernel function kx, ky, kz, a neural network hθ parameterized by θ, mini-batch size m, learning rate α. Output: parameter of classifier θ while θ has not converged do Sample a mini-batch of size m from input samples. Forward Propagation: calculate zi and hθ(x). Compute kernel matrices for X, Y and Zi using kx, ky, kz respectively inside mini-batch. Compute L(θ) via (6), where HSIC is evaluated empirically via (5). Backward Propagation: θ θ α L(θ). end |
| Open Source Code | Yes | Our code and adversarially robust models are publicly available.2 https://github.com/neu-spiral/HBa R |
| Open Datasets | Yes | We experiment with three standard datasets, MNIST [14], CIFAR-10 [12] and CIFAR-100 [12]. |
| Dataset Splits | No | No explicit mention of validation dataset splits (percentages or counts) or methodology to create them is found in the paper. |
| Hardware Specification | Yes | All experiments are carried out on a Tesla V100 GPU with 32 GB memory and 5120 cores. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., programming language, deep learning framework, or library versions) are mentioned in the paper. |
| Experiment Setup | Yes | Following prior literature, we set step size to 0.01 and radius r = 0.3 for MNIST, and step size as 2/255 and r = 8/255 for CIFAR-10 and CIFAR-100. ... We set λx as 1 and 0.006, λy as 50 and 0.05, for MNIST and CIFAR-10 respectively. |