Amnesiac Machine Learning
Authors: Laura Graves, Vineel Nagisetty, Vijay Ganesh11516-11524
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide extensive empirical analysis that show that these methods are indeed efficient, safe to apply, effectively remove learned information about sensitive data from trained models while maintaining model efficacy. We provide a detailed empirical evaluation of the efficacy of our methods along several vectors, including protection against data leaks, efficiency, and model performance. We conduct extensive experiments to evaluate the efficacy of our unlearning methods in comparison to the naive retraining method. |
| Researcher Affiliation | Academia | Laura Graves, Vineel Nagisetty, Vijay Ganesh University of Waterloo laura.graves@uwaterloo.ca, vineel.nagisetty@uwaterloo.ca, vganesh@uwaterloo.ca |
| Pseudocode | No | The paper describes the proposed methods, including mathematical formulations for Amnesiac Unlearning, but it does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code: Code used to run experiments can be found at https://github.com/lmgraves/Amnesiac ML |
| Open Datasets | Yes | Datasets: We conducted experiments on the following two well-known datasets. These datasets were chosen because of the ubiquity of experiments using them as well as to highlight the performance of our algorithm against tasks of varying complexity. 1. MNIST handwritten image dataset (Le Cun and Cortes 2010) is a widely used 10-class dataset consisting of 60,000 training images and 10,000 testing images. 2. CIFAR100 (Krizhevsky, Hinton et al. 2009) is a 100-class dataset consisting of 600 images from each class. |
| Dataset Splits | No | The paper specifies training and testing splits for MNIST (60,000 training, 10,000 testing) and describes how datasets were created for membership inference attacks, but it does not explicitly mention a separate validation set or how the data was split into training, validation, and test sets for the main model training. |
| Hardware Specification | Yes | All experiments were conducted on the Amazon Sagemaker platform using an ml.g4dn.xlarge instance with 4 v CPUs, 1 GPU, and 16GB of memory. |
| Software Dependencies | Yes | Our algorithms are implemented in Python 3.7 and use the PyTorch deep learning library (Paszke et al. 2019). |
| Experiment Setup | Yes | Our algorithms are implemented in Python 3.7 and use the PyTorch deep learning library (Paszke et al. 2019). Our experiments were performed on the Resnet18 convolutional neural network (He et al. 2016), a state-of-the-art residual learning architecture. In the case of naive retraining and unlearning, this is simply changing the datasets and has not had an effect, while in the case of amnesiac unlearning it represents the reversal of the batches containing the sensitive data. Subsequent membership inference attacks were performed after each epoch of training on the modified datasets. We found we had better results starting each inversion with a small amount of noise added to each feature, so each inversion is different. Finally, the original attack continues until the change in loss is below some threshold, while we found the attack to be more effective if we continued the attack for some set number of iterations, even while the change in loss was small. |