Fast Model DeBias with Machine Unlearning
Authors: Ruizhe Chen, Jianfei Yang, Huimin Xiong, Jianhong Bai, Tianxiang Hu, Jin Hao, YANG FENG, Joey Tianyi Zhou, Jian Wu, Zuozhu Liu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on the Colored MNIST, Celeb A, and Adult Income datasets along with experiments with large language models demonstrate that our method achieves superior or competing accuracies compared with state-of-the-art methods while attaining significantly fewer biases and requiring much less debiasing cost. Extensive experiments and detailed analysis on multiple datasets demonstrate that our framework can obtain competing accuracies with significantly smaller biases and much fewer data and computational costs. |
| Researcher Affiliation | Collaboration | Ruizhe Chen1, Jianfei Yang2, Huimin Xiong1, Jianhong Bai1, Tianxiang Hu1, Jin Hao3, Yang Feng4, Joey Tianyi Zhou5, Jian Wu1, Zuozhu Liu1 1 Zhejiang University 2 Nanyang Technological University 3 Stanford University 4 Angelalign Technology Inc. 5 Centre for Frontier AI Research |
| Pseudocode | Yes | Algorithm 1: The FMD framework. |
| Open Source Code | No | No explicit statement providing open-source code for the methodology described in this paper or a link to a code repository. |
| Open Datasets | Yes | Our experiments are conducted on three datasets. Colored MNIST is constructed by adding color bias to the MNIST dataset [68]. Celeb A [71] is a face recognition dataset... Adult Income Dataset is a publicly available dataset in the UCI repository [72]... As for the experiment on the language model, we use Stereo Set [76] as our test set. |
| Dataset Splits | Yes | Colored MNIST... The split of the training set, test set, and external set is 60000, 10000, and 10000. Celeb A... consists of 162,770 images for training and 9,867 images for testing. Adult Income Dataset... We split 200 samples from the test set as the external dataset. |
| Hardware Specification | Yes | The running time of all baselines is evaluated on a single RTX3090 GPU for a fair comparison. |
| Software Dependencies | No | We use pre-trained BERT [91] and GPT-2 [92], provided by Huggingface. |
| Experiment Setup | Yes | During training, we set the batch size of 256 for Colored MNIST and Adult, respectively, and 64 for Celeb A following [25, 78, 7]. We use pre-trained BERT [91] and GPT-2 [92], provided by Huggingface. During unlearning, we freeze the parameters of all other layers except the last classifier layer. In our experiment, we select the number of samples k=5000 for Colored MNIST, and k=200 for both Adult and Celeb A. The bias threshold is set to 0. |