Verification of Machine Unlearning is Fragile
Authors: Binchi Zhang, Zihan Chen, Cong Shen, Jundong Li
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate the efficacy of our methods through theoretical analysis and empirical experiments using real-world datasets. This study highlights the vulnerabilities and limitations in machine unlearning verification, paving the way for further research into the safety of machine unlearning. 5. Experiments In this section, we empirically evaluate the vulnerability of MUL verification with numerical experiments. |
| Researcher Affiliation | Academia | Binchi Zhang 1 Zihan Chen 1 Cong Shen 1 Jundong Li 1 1University of Virginia, Charlottesville, VA, USA. Correspondence to: Jundong Li <jundong@virginia.edu>. |
| Pseudocode | Yes | Algorithm 1 Retraining-based Adversarial Unlearning Algorithm |
| Open Source Code | Yes | Our code is available at https://github.com/zhangbinchi/ unlearning-verification-is-fragile. |
| Open Datasets | Yes | Our experiments are based on three widely adopted real-world datasets for image classification, MNIST (Le Cun et al., 1998), SVHN (Netzer et al., 2011), and CIFAR-10 (Krizhevsky et al., 2009): the MNIST dataset consists of a collection of handwritten digit images; the CIFAR-10 dataset contains color images in 10 classes, with each class representing a specific object category, e.g., cats and automobiles; the SVHN dataset consists of house numbers images captured from Google Street View. The statistics of these three datasets are shown in Table 1. All datasets are publicly accessible (MNIST with GNU General Public License, CIFAR-10 with MIT License, and SVHN with CC BY-NC License). |
| Dataset Splits | Yes | All datasets utilized in our experiments adhere to the standard train/test split provided by the Torchvision library (maintainers & contributors, 2016). Within each experiment, 20% of the training data is set aside as the validation set. |
| Hardware Specification | Yes | All experiments were conducted on an Nvidia RTX A6000 GPU. |
| Software Dependencies | No | The paper states 'We implemented all experiments in the PyTorch (Paszke et al., 2019) library and exploited SGD as the optimizer for training.' and refers to 'Torchvision library (maintainers & contributors, 2016)'. However, it does not specify version numbers for PyTorch or Torchvision. |
| Experiment Setup | Yes | For consistent hyperparameter settings across all datasets, we fix the learning rate γ(t) as 10-2, the weight decay parameter as 5 10-4, the training epochs number as 30, and set the batch size to 128. |