Knowledge Removal in Sampling-based Bayesian Inference
Authors: Shaopeng Fu, Fengxiang He, Dacheng Tao
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on Gaussian mixture models and Bayesian neural networks confirm the effectiveness of the proposed algorithm. The code is available at https: //github.com/fshp971/mcmc-unlearning. ... In this section, we empirically verify the effectiveness and efficiency of the proposed MCMC unlearning algorithm on the Gaussian mixture models and Bayesian neural networks. |
| Researcher Affiliation | Collaboration | Shaopeng Fu1 , Fengxiang He2,1 & Dacheng Tao1 1The University of Sydney, Australia, 2JD Explore Academy, China |
| Pseudocode | Yes | Algorithm 1 (MCMC unlearning). Suppose one have drawn a series of samples {θ1, , θT } from the posterior p S via MCMC. Then, MCMC unlearning algorithm removes the learned knowledge of dataset S from each drawn sample θi as follows, θ i θi I(S ), where I(S ) is the MCMC influence function for dataset S S. |
| Open Source Code | Yes | The code is available at https: //github.com/fshp971/mcmc-unlearning. |
| Open Datasets | Yes | We employ two image datasets, Fashion-MNIST (Xiao et al., 2017) and CIFAR-10 (Krizhevsky et al., 2009), in our experiments. |
| Dataset Splits | No | The paper describes dividing the training set into a 'remained part Sr' and a 'removed part Sf', and also mentions a 'test set Stest'. However, it does not explicitly provide details about a separate validation set or its split for hyperparameter tuning or early stopping. |
| Hardware Specification | No | The paper does not provide specific details regarding the hardware used for running experiments, such as GPU or CPU models, processor types, or memory amounts. |
| Software Dependencies | No | The paper mentions software frameworks like 'Py Torch (Paszke et al., 2017)' and 'Tensor Flow (Abadi et al., 2015)' but does not specify their version numbers or other ancillary software dependencies with versions. |
| Experiment Setup | Yes | Every BNN is trained for 10,000 iterations. The batch size is set as 128. For both SGLD and SGHMC, we first train the model without noise injection in the first 1,000 iterations. In this stage, the learning rate is fixed to 0.01/N. Then, we resume the traditional SGLD and SGHMC in the rest of the training. In this stage, the learning rate schedule is set as 0.01 t 0.5005/N, where t is the training iteration step. Besides, the momentum factor α of SGHMC is set as 0.9. ... When calculating the inversed-Hessian-vector product H 1v in the MCMC influence function (see Appendix B), the recursive calculation number j is set as 64, and the scaling factor c is set as 0.05/N , in which N is the number of the current remained training examples. |