Variational Open-Domain Question Answering
Authors: Valentin Liévin, Andreas Geert Motzfeldt, Ida Riis Jensen, Ole Winther
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4. Experiments In this section, we present the medical domain tasks and datasets, results on end-to-end multiple-choice ODQA and its application to information retrieval. |
| Researcher Affiliation | Collaboration | 1Section for Cognitive Systems, Technical University of Denmark, Denmark 2Find Zebra, Denmark 3Center for Genomic Medicine, Rigshospitalet, Copenhagen University Hospital, Denmark 4Bioinformatics Centre, Department of Biology, University of Copenhagen, Denmark. |
| Pseudocode | No | The paper describes its methodology through mathematical formulations and textual explanations but does not include any explicitly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | Our implementation of the sampling methods and the VOD objective is available at http://github.com/Vod LM/vod. |
| Open Datasets | Yes | We release the Med Wiki corpus (under MIT license): a collection of 4.5% of articles taken from the English Wikipedia and targeted to the Med MCQA and USMLE datasets. |
| Dataset Splits | Yes | Table 2. Summarizes the medical QA datasets and corpora used in our study, including the Med MCQA, USMLE, and Find Zebra (FZ) corpus, with the Med Wiki as the knowledge base for all QA tasks. The questions are numbered for the train/validation/test splits. DATASETS MEDMCQA QUESTIONS 182.8K/4.2K/6.1K USMLE QUESTIONS 10.2K/1.3K/1.3K |
| Hardware Specification | Yes | All experiments were conducted on a single node of 8 RTX 5000 GPUs using half-precision. |
| Software Dependencies | Yes | Software Py Torch (Paszke et al., 2019) Lightning (Falcon) faiss (Johnson et al., 2021) BM25 implementation elasticsearch v7.14.1 faiss v1.7.2 |
| Experiment Setup | Yes | Table 12. Hyperparameters used across the multiple-choice ODQA experiments. Category Parameter Value Optimization Learning rate 3 10 6 Weight decay 1 10 3 Batching batch-size 32 K (documents per option) 8 P (retriever support size) 100 Training steps (Med MCQA) 150k Posterior and retrieval parameterization τ (BM25 temperature) 5 |