Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Knowledge Removal in Sampling-based Bayesian Inference
Authors: Shaopeng Fu, Fengxiang He, Dacheng Tao
ICLR 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on Gaussian mixture models and Bayesian neural networks confirm the effectiveness of the proposed algorithm. The code is available at https: //github.com/fshp971/mcmc-unlearning. ... In this section, we empirically verify the effectiveness and efficiency of the proposed MCMC unlearning algorithm on the Gaussian mixture models and Bayesian neural networks. |
| Researcher Affiliation | Collaboration | Shaopeng Fu1 , Fengxiang He2,1 & Dacheng Tao1 1The University of Sydney, Australia, 2JD Explore Academy, China |
| Pseudocode | Yes | Algorithm 1 (MCMC unlearning). Suppose one have drawn a series of samples {θ1, , θT } from the posterior p S via MCMC. Then, MCMC unlearning algorithm removes the learned knowledge of dataset S from each drawn sample θi as follows, θ i θi I(S ), where I(S ) is the MCMC influence function for dataset S S. |
| Open Source Code | Yes | The code is available at https: //github.com/fshp971/mcmc-unlearning. |
| Open Datasets | Yes | We employ two image datasets, Fashion-MNIST (Xiao et al., 2017) and CIFAR-10 (Krizhevsky et al., 2009), in our experiments. |
| Dataset Splits | No | The paper describes dividing the training set into a 'remained part Sr' and a 'removed part Sf', and also mentions a 'test set Stest'. However, it does not explicitly provide details about a separate validation set or its split for hyperparameter tuning or early stopping. |
| Hardware Specification | No | The paper does not provide specific details regarding the hardware used for running experiments, such as GPU or CPU models, processor types, or memory amounts. |
| Software Dependencies | No | The paper mentions software frameworks like 'Py Torch (Paszke et al., 2017)' and 'Tensor Flow (Abadi et al., 2015)' but does not specify their version numbers or other ancillary software dependencies with versions. |
| Experiment Setup | Yes | Every BNN is trained for 10,000 iterations. The batch size is set as 128. For both SGLD and SGHMC, we first train the model without noise injection in the first 1,000 iterations. In this stage, the learning rate is fixed to 0.01/N. Then, we resume the traditional SGLD and SGHMC in the rest of the training. In this stage, the learning rate schedule is set as 0.01 t 0.5005/N, where t is the training iteration step. Besides, the momentum factor α of SGHMC is set as 0.9. ... When calculating the inversed-Hessian-vector product H 1v in the MCMC influence function (see Appendix B), the recursive calculation number j is set as 64, and the scaling factor c is set as 0.05/N , in which N is the number of the current remained training examples. |