Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Position: Key Claims in LLM Research Have a Long Tail of Footnotes
Authors: Anna Rogers, Sasha Luccioni
ICML 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | This position paper argues that LLM research should be more precise with its key terms and claims. |
| Researcher Affiliation | Collaboration | 1IT University of Copenhagen 2Hugging Face, Canada. Correspondence to: Anna Rogers <EMAIL>, Sasha Luccioni <EMAIL>. |
| Pseudocode | No | No, the paper is a position paper and does not include any pseudocode or algorithm blocks. |
| Open Source Code | No | No, the paper is a position paper and does not describe a new methodology for which open-source code would be provided. |
| Open Datasets | No | No, this is a position paper and does not involve training models on a dataset. It discusses concepts and critiques claims within LLM research rather than presenting new experimental data. |
| Dataset Splits | No | No, this paper is a position paper and does not describe experiments that would involve dataset splits for validation. |
| Hardware Specification | No | No, the paper is a position paper and does not conduct experiments requiring specific hardware specifications. |
| Software Dependencies | No | No, the paper is a position paper and does not conduct experiments requiring specific software dependencies. |
| Experiment Setup | No | No, the paper is a position paper and does not describe an experimental setup with hyperparameters or system-level training settings. |