Forget Unlearning: Towards True Data-Deletion in Machine Learning
Authors: Rishav Chourasia, Neil Shah
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our solution achieves all the three objectives while providing substantial computation savings for both convex and non-convex losses. Notably, we demonstrate a powerful synergy between deletion privacy and differential privacy, where the same noise needed for privacy of present database records also ensures the erasure of information related to the deleted records. For convex and smooth losses, we certify that under a (q, εdd)-Rényi non-adaptive deletion-privacy and (q, εdp)-Rényi differential-privacy constraint, our Noisy GD based data-deletion mechanism for d-dimensional models over n-sized databases with requests that modify up to r records at a time can maintain the pareto-optimal excess empirical risk of the order O qd εdpn2 while being Ω(n log(min{ n qd }) cheaper than retraining in gradient complexity. For non-convex, bounded and smooth losses, we show a computational saving of Ω(dn log n r ) in gradient complexity under the same constraints with an excess risk of O qd εdpn2 + 1 εdp . |
| Researcher Affiliation | Academia | Rishav Chourasia 1 Neil Shah 1 1Department of Computer Science, National University of Singapore, Singapore. Correspondence to: Rishav Chourasia <rishav1@comp.nus.edu.sg>. |
| Pseudocode | Yes | Algorithm 1 Interacting curator (A, A, fpub) & requester Q; Algorithm 2 Noisy-GD: Noisy Gradient Descent |
| Open Source Code | No | The paper does not provide any explicit statements about releasing source code or links to a code repository. |
| Open Datasets | No | The paper does not mention the use of any specific named public datasets (e.g., MNIST, CIFAR-10) or provide links/citations for accessing any datasets. It refers to abstract "databases". |
| Dataset Splits | No | The paper does not specify exact dataset splits (e.g., training, validation, test percentages or counts) or reference any predefined splits with citations. |
| Hardware Specification | No | The paper does not mention any specific hardware (e.g., GPU/CPU models, cloud instances, or workstation specifications) used for running its experiments. |
| Software Dependencies | No | The paper does not list specific software components with their version numbers (e.g., Python 3.8, PyTorch 1.9, CUDA 11.1). |
| Experiment Setup | Yes | Theorem 5.1 (Accuracy, privacy, deletion, and computation tradeoffs) states: "If the learning rate is η = 1 2(λ+β), the gradient noise variance is σ2 = 4q L2 λεdpn2 , and the weight initialization distribution is ρ = N 0, σ2 λ(1 ηλ/2)Id ". It also specifies conditions for KA and K A which are numbers of iterations. |