Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Can Pruning Improve Certified Robustness of Neural Networks?
Authors: Zhangheng LI, Tianlong Chen, Linyi Li, Bo Li, Zhangyang Wang
TMLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically verify our findings with off-the-shelf pruning methods and further present a new stability-based pruning method tailored for reducing neuron instability, that outperforms existing pruning methods in enhancing certified robustness. Our experiments show that by appropriately pruning an NN, its certified accuracy can be boosted up to 8.2% under standard training, and up to 24.5% under adversarial training on the CIFAR10 dataset. |
| Researcher Affiliation | Academia | Department of Electrical and Computer Engineering University of Texas at Austin; Department of Computer Science University of Illinois Urbana-Champaign |
| Pseudocode | No | The paper describes methodologies and processes using prose and mathematical equations (e.g., Eq. 1, Eq. 2, Eq. 3, Eq. 4, Eq. 5) but does not include any explicitly labeled pseudocode blocks or algorithms. |
| Open Source Code | No | Codes will be fully released. |
| Open Datasets | Yes | Across our experiments, we choose 3 classification datasets, Fashion MNIST Xiao et al. (2017), SVHN Netzer et al. (2011), and CIFAR10 Krizhevsky et al. (2009) as the benchmark datasets. We also conduct experiments on the downscaled 64x64 Image Net Chrabaszcz et al. (2017) |
| Dataset Splits | Yes | Fashion MNIST: Fashion MNIST is an MNIST-like greyscale image classification dataset by replacing hand-written digits with fashion items, which are more difficult to classify. It has a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. We use the first 200 samples from the testing dataset for verification. SVHN: SVHN is a dataset consisting of Street View House Number images, with each image consisting of a single cropped digit labeled from 0 to 9. Each example is a 32x32 RGB image. We use the first 200 samples from the testing dataset for verification. CIFAR10: CIFAR10 is a dataset consisting of 10 object classes in the wild, and each class has 6000 samples. This dataset is commonly used in prior works in complete verification, following Wang et al. (2021), we choose the ERAN test set Singh et al. (2019) which consists of 1000 images from the CIFAR10 test set. Note that we only use the first 200 samples in the ERAN test set for verification efficiency. |
| Hardware Specification | Yes | We run the verifications using one NVIDIA RTX A6000 GPU card. This model has 6.7M parameters and is the largest network that can be fitted in a GPU with 24GB memory for complete verification and can also achieve good baseline performance on our chosen benchmarks. |
| Software Dependencies | No | The paper mentions several frameworks, methods, and optimizers used (e.g., FGSM+Grad Align, auto-Li RPA, Beta-CROWN, Adam, Stochastic Gradient Descent, IBP). However, it does not specify version numbers for any core software libraries or programming languages (e.g., Python, PyTorch, TensorFlow, CUDA). |
| Experiment Setup | Yes | The batch size is set to 128, and we clip the norm of gradients to a maximum of 8. We train 200 epochs in one pruning iteration for each experiment, with a learning rate decay factor of 0.1 at 140 and 170 epochs. We set the input perturbation ϵ as Linf-norm ball to 0.1 for the Fashion MNIST dataset and 2/255 for SVHN and CIFAR10 datasets, and gradually increase ϵ from 0 to 2/255 starting from 11th epoch and until 80th epoch. We also scale the perturbation to 8/255 to validate the effectiveness of pruning under bigger perturbations. The training epochs under 8/255 perturbation is set to 300. After each pruning iteration, we rewind the remaining weights to initial states and reset the optimizer with the initial learning rate and ϵ. For adversarial training: The learning rate is set to 0.01, and we use Stochastic Gradient Descent with 0.9 momentum and 0.0005 weight decay as optimizer. For certified training: The learning rate is set to 0.001, and we use Adam with a weight decay of 0 for RSLoss and NRSLoss-based pruning and 0.00001 for other pruning methods. |