Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
RUBi: Reducing Unimodal Biases for Visual Question Answering
Authors: Remi Cadene, Corentin Dancette, Hedi Ben younes, Matthieu Cord, Devi Parikh
NeurIPS 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We run extensive experiments on VQA-CP v2 [10] and demonstrate the ability of RUBi to surpass current state-of-the-art results from a significant margin. |
| Researcher Affiliation | Collaboration | 1 Sorbonne Université, CNRS, LIP6, 4 place Jussieu, 75005 Paris, 2 Facebook AI Research, 3 Georgia Institute of Technology |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available: github.com/cdancette/rubi.bootstrap.pytorch |
| Open Datasets | Yes | We train and evaluate our models on VQA-CP v2 [10]. ... We also evaluate our models on the standard VQA v2 dataset [9]. |
| Dataset Splits | Yes | We train and evaluate our models on VQA-CP v2 [10]. This dataset was developed to evaluate the models robustness to question biases. We follow the same training and evaluation protocol as [25], who also propose a learning strategy to reduce biases. For each model, we report the standard VQA evaluation metric [8]. We also evaluate our models on the standard VQA v2 [9]." and "VQA v2 train, val and test sets follow the same distribution |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions software components like 'Faster R-CNN' and 'GRU' but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | No | The paper states 'Further implementation details are included in the supplementary materials,' indicating that specific experimental setup details, such as hyperparameters, are not provided in the main text. |