Mitigating Biases in Blackbox Feature Extractors for Image Classification Tasks
Authors: Abhipsa Basu, Saswat Subhajyoti Mallick, Venkatesh Babu R
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments demonstrate the effectiveness of our method across multiple benchmarks. The code is publicly available at https: //github.com/abhipsabasu/blackbox_bias_mitigation. and Extensive experiments show that the proposed method is effective across multiple benchmarks. |
| Researcher Affiliation | Academia | Abhipsa Basu Saswat Subhajyoti Mallick R. Venkatesh Babu Vision and AI Lab, Indian Institute of Science, Bangalore and Work done while working as a project assistant at the Indian Institute of Science |
| Pseudocode | No | The paper describes its method in text and figures but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is publicly available at https: //github.com/abhipsabasu/blackbox_bias_mitigation. |
| Open Datasets | Yes | Waterbirds [23] is a dataset of birds..., The Color MNIST dataset (CMNIST) [72] is generated from MNIST [73]..., The real-world Celeb A dataset [64] consists of 202, 599 face images..., UTKFace [74] and BAR [20]. |
| Dataset Splits | Yes | We assume the availability of a small group-balanced (but unannotated) validation set and calculate the overall accuracy over this dataset for model selection during the bias-mitigation phase. and We use the first part as the training set, second part as test, and third part as the validation set. |
| Hardware Specification | Yes | It is to be noted that we perform all our experiments on a single Nvidia-RTX A5000 GPU. |
| Software Dependencies | No | The paper does not explicitly state specific version numbers for software dependencies (e.g., Python, PyTorch, CUDA) required for replication. |
| Experiment Setup | Yes | Hyperparameter tuning happens via the validation accuracy of the model checkpoint saved during the mitigation stage. The bias-amplification stage has the following hyperparameters: LR (learning rate), BS (batch size), λ (weight decay) and number of epochs. In the clustering stage, the number of clusters K is a hyperparameter. Finally, in the mitigation stage, we have two hyperparameters specific to the margin loss: the scaling parameter s (i.e. the radius of the hypersphere on which the features are projected) and the standard deviation for the Gaussian randomization, σ. We next define the range of each hyperparamter on which we evaluate our model and the other methods. For learning rate LR, we explore in the range 0.0001, 0.0005, , 0.05 in step sizes of 5 and 2 respectively. Similarly, for batch size BS we evaluate on the range {64, 128, 256, 512}. We explore higher values of weight decay λ {1, 0.1, 0.05, 0.01} for the bias-amplication stage and for the mitigation stage, we search λ in the range 10 6, , 10 2 in step sizes of 10 and also consider λ = 0. Number of epochs for training is kept in the range {50, 100}. For the clustering stage, we choose K {2, 4, 6, 8} for Waterbirds and Celeb A, and explore K = 10, 20, , 60 with a step size of 10 for the CMNIST variants. In the mitigation stage, we select s {4, 8, 12, 16} and σ {0, 0.05, 0.1, 0.15, 0.2}. We show the selected hyperparameters in the bias-amplification stage in Tables 9 and 10, and for the mitigation procedure, the chosen hyperparameters are present in Table 11. For number of neurons M in the MLP layer, we fix the value to be 128. For Celeb A experiments, we use SGD as the optimizer, whereas for Waterbirds, and both the variants of CMNIST, we use Adam. |