Can Implicit Bias Imply Adversarial Robustness?
Authors: Hancheng Min, Rene Vidal
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Moreover, numerical experiments on real datasets show that shallow networks with our generalized Re LU activation functions are much more robust than those with a Re LU activation. |
| Researcher Affiliation | Academia | 1University of Pennsylvania, Philadelphia, PA, USA. Correspondence to: Hancheng Min <hanchmin@seas.upenn.edu>. |
| Pseudocode | Yes | Algorithm 1 Estimating dist(fp, F) |
| Open Source Code | Yes | Code available at https://github.com/hanchmin/pReLU_ICML24. |
| Open Datasets | Yes | We first consider training a p Re LU network of width h = 500 to predict whether an MNIST digit is even or odd. |
| Dataset Splits | No | We run SGD (batch size 100 and step size 0.2 for 2 105 epochs) with small initialization (all weights initialized as mean-zero Gaussian with standard deviation 10 7) to train a p Re LU network with h = 2000 neurons on a dataset drawn from DX,Y with D = 1000, K = 10, and K1 = 6. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory, or cloud services) are mentioned in the paper. |
| Software Dependencies | No | The paper mentions optimizers like "SGD" and "Adam," but does not provide specific version numbers for any software, programming languages, or libraries used for implementation. |
| Experiment Setup | Yes | We run SGD (batch size 100 and step size 0.2 for 2 105 epochs) with small initialization (all weights initialized as mean-zero Gaussian with standard deviation 10 7) to train a p Re LU network with h = 2000 neurons on a dataset drawn from DX,Y with D = 1000, K = 10, and K1 = 6. |