Learning from Failure: De-biasing Classifier from Biased Classifier
Authors: Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, Jinwoo Shin
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that our method significantly improves the training of network against various types of biases in both synthetic and real-world datasets. Surprisingly, our framework even occasionally outperforms the debiasing methods requiring explicit supervision of the spuriously correlated attributes. |
| Researcher Affiliation | Academia | 1School of Electrical Engineering, KAIST 2Graduate School of AI, KAIST |
| Pseudocode | Yes | Algorithm 1 Learning from Failure |
| Open Source Code | No | The paper mentions 'BAR is publicly available1' with a footnote linking to 'https://github.com/alinlab/BAR', which is specified as a dataset. There is no explicit statement or link provided for the open-source code of the Lf F methodology itself. |
| Open Datasets | Yes | We show the effectiveness of Lf F on various biased datasets, including Colored MNIST [14, 18] with color bias, Corrupted CIFAR-10 [12] with texture bias, and Celeb A [19] with gender bias. In addition, we newly construct a real-world dataset, coined biased action recognition (BAR), to resolve the lack of realistic evaluation benchmark for debiasing schemes. BAR is publicly available1, and a detailed description of BAR is in Appendix B. |
| Dataset Splits | No | The paper describes the construction of 'unbiased evaluation set' and 'bias-conflicting evaluation set' and varies the ratio of bias-aligned samples in the 'training dataset', but it does not provide specific training/validation/test splits (e.g., percentages or counts) for model training and hyperparameter tuning processes. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU models, CPU types, or memory) used for running the experiments. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers (e.g., specific versions of deep learning frameworks or libraries). |
| Experiment Setup | No | The paper mentions hyperparameters like 'q' for GCE loss and 'learning rate η, number of iterations T' in Algorithm 1. It also states 'We provide a detailed description of datasets we considered in Appendix A, B and experimental details in Appendix C.' in Section 4. However, specific numerical values for these hyperparameters and detailed training configurations are not provided in the main text of the paper. |