BNN-DP: Robustness Certification of Bayesian Neural Networks via Dynamic Programming
Authors: Steven Adams, Andrea Patane, Morteza Lahijanian, Luca Laurenti
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | On a set of experiments on various regression and classification tasks and BNN architectures, we show that BNN-DP outperforms state-of-the-art methods by up to four orders of magnitude in both tightness of the bounds and computational efficiency.We validate our framework on several regression and classification tasks, including the Kin8nm, MNIST, Fashion MNIST, and CIFAR-10 datasets, and a range of BNN architectures. |
| Researcher Affiliation | Academia | Delft Center for Systems and Control, Technical University of Delft, Delft, 2628 CD, The Netherlands School of Computer Science and Statistics, Trinity College Dublin, Dublin 2, Ireland Departement of Aerospace Engineering Sciences and Computer Science, University of Colorado Boulder, Boulder, CO 80303, USA. |
| Pseudocode | Yes | Algorithm 1 Adversarial Robustness for Classification; Algorithm 2 Interval relaxation procedure for Eqn. 5a with h = softmax; Algorithm 3 Back-Propagation of PWA Relaxations.; Algorithm 4 Adversarial Robustness for Regression |
| Open Source Code | Yes | Our code is available at https://github.com/ sjladams/BNN_DP. |
| Open Datasets | Yes | We validate our framework on several regression and classification tasks, including the Kin8nm, MNIST, Fashion MNIST, and CIFAR-10 datasets, and a range of BNN architectures. The latter dataset contains state-space readings for the dynamics of an 8 link robot arm, and is commonly used as a regression task to benchmark BNNs (Hern andez-Lobato & Adams, 2015; Gal & Ghahramani, 2016).10Available at http://www.cs.toronto.edu/ delve. |
| Dataset Splits | No | The paper refers to testing on '100 test points' and '100 randomly sampled test points' but does not explicitly provide training, validation, and test dataset splits with percentages or sample counts. |
| Hardware Specification | Yes | We ran our experiments on an AMD EPYC 7252 8-core CPU and train the BNNs using Noisy Adam (Zhang et al., 2018a) and variational online Gauss-Newton (Khan et al., 2018). |
| Software Dependencies | No | The paper mentions using 'Noisy Adam' and 'variational online Gauss-Newton' as training methods but does not list specific software packages with version numbers (e.g., PyTorch 1.9, TensorFlow 2.x). |
| Experiment Setup | No | The paper mentions training methods (Noisy Adam, variational online Gauss-Newton) and BNN architectures, but does not provide specific hyperparameter values such as learning rates, batch sizes, or number of epochs. |