Expressive Losses for Verified Robustness via Convex Combinations
Authors: Alessandro De Palma, Rudy R Bunel, Krishnamurthy Dj Dvijotham, M. Pawan Kumar, Robert Stanforth, Alessio Lomuscio
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present a comprehensive experimental evaluation of CC-IBP, MTL-IBP and Exp-IBP on benchmarks from the robust vision literature, demonstrating that the three methods attain state-of-the-art performance in spite of their conceptual simplicity. We now present experimental results that support the centrality of expressivity, as defined in 3, to verified training. |
| Researcher Affiliation | Collaboration | Alessandro De Palma1a, Rudy Bunel2, Krishnamurthy (Dj) Dvijotham2, M. Pawan Kumar2, Robert Stanforth2, Alessio Lomuscio3 1Inria, École Normale Supérieure, PSL University, CNRS 2Google Deep Mind 3Imperial College London |
| Pseudocode | Yes | Pseudo-code for both losses can be found in appendix D. Algorithm 1 provides pseudo-code for CC-MTL, MTL-IBP and Exp-IBP. |
| Open Source Code | Yes | Code is available at https://github.com/alessandrodepalma/expressive-losses. |
| Open Datasets | Yes | We benchmark on the CIFAR10 (Krizhevsky & Hinton, 2009), Tiny Image Net (Le & Yang, 2015), and downscaled Image Net (64 64) (Chrabaszcz et al., 2017) datasets. MNIST (Le Cun et al., 2010) is a 10-class (each class being a handwritten digit) classification dataset consisting of 28 28 greyscale images, with 60,000 training and 10,000 test points. |
| Dataset Splits | Yes | We train on the entire training sets, and evaluate MNIST and CIFAR-10 on their test sets, Image Net and Tiny Image Net on their validation sets. |
| Hardware Specification | Yes | In particular, we relied on the following CPU models: Intel i7-6850K, i9-7900X CPU, i9-10920X, i9-10940X, AMD 3970X. The employed GPU models for most of the experiments were: Nvidia Titan V, Titan XP, and Titan XP. The experiments of 6.3 and appendix G.5 on CIFAR-10 for ϵ = 8/255 were run on the following GPU models: RTX 4070 Ti, RTX 2080 Ti, RTX 3090. |
| Software Dependencies | No | The paper mentions several software components like PyTorch, automatic Li RPA, OVAL framework, and torchvision.datasets, but does not provide specific version numbers for them (e.g., 'We implemented CC-IBP, MTL-IBP and Exp-IBP in Py Torch'). |
| Experiment Setup | Yes | Details pertaining to the employed datasets, hyper-parameters, network architectures, and computational setup are reported in appendix F. Table 3: Hyper-parameter configurations for the networks trained via CC-IBP, MTL-IBP and Exp-IBP from tables 1 and 4. |