Learning Adversarially Robust Representations via Worst-Case Mutual Information Maximization
Authors: Sicheng Zhu, Xiao Zhang, David Evans
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on downstream classification tasks support the robustness of the representations found using unsupervised learning with our training principle. |
| Researcher Affiliation | Academia | 1Department of Computer Science, University of Virginia. |
| Pseudocode | Yes | More specifically, our algorithm alternatively performs gradient ascent on θ for the inner maximization problem of estimating b Im(X ; g(X )) given µ X, and searches for the set of worst-case perturbations on {x i : i [m]} given θ based on projected gradient descent. (see Appendix B for the pseudocode and a complexity analysis of the proposed algorithm). |
| Open Source Code | Yes | https://github.com/schzhu/learning-adversarially-robust-representations |
| Open Datasets | Yes | We conduct experiments on MNIST (Le Cun & Cortes, 2010), Fashion-MNIST (Xiao et al., 2017), SVHN (Netzer et al., 2011), and CIFAR-10 (Krizhevsky et al., 2009) |
| Dataset Splits | No | The paper mentions using standard datasets but does not explicitly provide the training/validation/test split percentages or sample counts for reproduction. While standard datasets have predefined splits, the paper does not state how they were used. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions using MINE and leveraging implementations from Engstrom et al. (2019a) and Hjelm et al. (2018) but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | We conduct experiments on MNIST (Le Cun & Cortes, 2010), Fashion-MNIST (Xiao et al., 2017), SVHN (Netzer et al., 2011), and CIFAR-10 (Krizhevsky et al., 2009), considering typical ℓ -norm bounded adversarial perturbations for each dataset (ϵ = 0.3 for MNIST, 0.1 for Fashion MNIST, 4/255 for SVHN, and 8/255 for CIFAR-10). We use the PGD attack (M adry et al., 2018) for both generating adversarial distributions in the estimation of worstcase mutual information and evaluating model robustness. To implement our proposed estimator (4.3), we adopt the encode-and-dot-product model architecture in Hjelm et al. (2018) and adjust it to adapt to different forms of representations. We leverage implementations from Engstrom et al. (2019a) and Hjelm et al. (2018) in our implementation. Implementation details are provided in Appendix D.1. |