A Robust Optimisation Perspective on Counterexample-Guided Repair of Neural Networks
Authors: David Boetius, Stefan Leue, Tobias Sutter
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically study the practical implications of our theoretical results, demonstrating the suitability of common verifiers and falsifiers for repair despite a disadvantageous theoretical result. |
| Researcher Affiliation | Academia | 1Department of Computer and Information Science, University of Konstanz, Konstanz, Baden-W urttemberg, Germany. |
| Pseudocode | Yes | Algorithm 1 Counterexample-Guided Repair |
| Open Source Code | Yes | Our source code is available at https://github.com/sen-uni-kn/ specrepair. |
| Open Datasets | Yes | The MNIST dataset (Le Cun et al., 1998) consists of 70 000 labelled images of hand-written Arabic digits. ... The ACAS Xu networks (Katz et al., 2017) form a collision avoidance system for aircraft without onboard personnel. ... The Collision Detection dataset (Ehlers, 2017) is introduced for evaluating neural network verifiers. ... We use Recursive Model Indices (RMIs) (Kraska et al., 2018) in our experiments. |
| Dataset Splits | Yes | To replace the unavailable ACAS Xu training data, we randomly sample a training and a validation set and compare with the scores produced by the original network. |
| Hardware Specification | Yes | The ACAS Xu, Collision Detection and Integer Dataset RMI experiments were run on a compute server with an Intel Xeon E5 2580 v4 2.4GHz CPU (28 cores) and 1008GB of memory. The MNIST experiments were run on a GPU compute server with an AMD Ryzen Threadripper 3960X 24-Core Processor and 252GB of memory, utilising an NVIDIA RTX A6000 GPU with 48GB of memory. |
| Software Dependencies | Yes | All experiments were conducted on Ubuntu 2022.04.1 LTS machines using Python 3.8. ... Spec Repair is based on Py Torch (Paszke et al., 2019). ... The quadratic programming repair algorithm for linear regression models is implemented in Python and leverages Gurobi (Gurobi Optimization, LLC, 2021). |
| Experiment Setup | Yes | We train the MNIST network using Stochastic Gradient Descent (SGD) with a mini-batch size of 32, a learning rate of 0.01 and a momentum coefficient of 0.9, training for two epochs. Counterexample-removal uses the same setup, except for using a decreased learning rate of 0.001 and iterating only for a tenth of an epoch. |