FairCal: Fairness Calibration for Face Verification
Authors: Tiago Salvador, Stephanie Cairns, Vikram Voleti, Noah Marshall, Adam M Oberman
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we introduce the Fairness Calibration (Fair Cal) method, a post-training approach that simultaneously: (i) increases model accuracy (improving the stateof-the-art), (ii) produces fairly-calibrated probabilities, (iii) significantly reduces the gap in the false positive rates, (iv) does not require knowledge of the sensitive attribute, and (v) does not require retraining, training an additional model, or retuning. We apply it to the task of Face Verification, and obtain state-of-the-art results with all the above advantages. |
| Researcher Affiliation | Academia | Tiago Salvador1,3, Stephanie Cairns1,3, Vikram Voleti2,3, Noah Marshall1,3, Adam Oberman1,3 1 Mc Gill University 2 Université de Montréal 3 Mila |
| Pseudocode | No | The paper describes the Fair Cal method in prose and mathematical equations in Section 4.1 but does not include a formal pseudocode or algorithm block. |
| Open Source Code | Yes | All methods were implemented in Python, and the code is provided in the supplemental material. |
| Open Datasets | Yes | We present experiments on two different datasets: Racial Faces in the Wild (RFW) (Wang et al., 2019a) and Balanced Faces in the Wild (BFW) (Robinson et al., 2020), both of which are available under licenses for non-commercial research purposes only. ... The RFW and BFW datasets are made up of images taken from MS-Celeb-1M (Guo et al., 2016) and VGGFace (Cao et al., 2018), respectively. |
| Dataset Splits | Yes | The results we present are the product of leave-one-out cross-validation. |
| Hardware Specification | Yes | The embeddings from the pretrained models were obtained on a machine with one Ge Force GTX 1080 Ti GPU. |
| Software Dependencies | No | The reproducibility statement mentions "All methods were implemented in Python" but does not specify a version. It also mentions "ADAM optimizer" but no library or version associated with it. |
| Experiment Setup | Yes | For both the FSN and Fair Cal method we used K = 100 clusters for the K-means algorithm, as recommended by Terhörst et al. (2020b). For Fair Cal, we employed the recently proposed beta calibration method (Kull et al., 2017). ... For AGENDA: "All training is done with a batch size of 400 and an ADAM optimizer with a learning rate of 10^3." For FTC: "The network was trained with a batchsize of b = 200 over 50 epochs, using an Adam optimizer with a learning rate of 10^-3 and weight decay of 10^-3." |