Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
RUBi: Reducing Unimodal Biases for Visual Question Answering
Authors: Remi Cadene, Corentin Dancette, Hedi Ben younes, Matthieu Cord, Devi Parikh
NeurIPS 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We run extensive experiments on VQA-CP v2 [10] and demonstrate the ability of RUBi to surpass current state-of-the-art results from a significant margin. |
| Researcher Affiliation | Collaboration | 1 Sorbonne Université, CNRS, LIP6, 4 place Jussieu, 75005 Paris, 2 Facebook AI Research, 3 Georgia Institute of Technology |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available: github.com/cdancette/rubi.bootstrap.pytorch |
| Open Datasets | Yes | We train and evaluate our models on VQA-CP v2 [10]. ... We also evaluate our models on the standard VQA v2 dataset [9]. |
| Dataset Splits | Yes | We train and evaluate our models on VQA-CP v2 [10]. This dataset was developed to evaluate the models robustness to question biases. We follow the same training and evaluation protocol as [25], who also propose a learning strategy to reduce biases. For each model, we report the standard VQA evaluation metric [8]. We also evaluate our models on the standard VQA v2 [9]." and "VQA v2 train, val and test sets follow the same distribution |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions software components like 'Faster R-CNN' and 'GRU' but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | No | The paper states 'Further implementation details are included in the supplementary materials,' indicating that specific experimental setup details, such as hyperparameters, are not provided in the main text. |