Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Power of Ordered Hypothesis Testing
Authors: Lihua Lei, William Fithian
ICML 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We compare these methods using the GEOQuery data set analyzed by (Li & Barber, 2015) and find Adaptive Seq Step has favorable performance for both good and bad prior orderings. . . . In Section 5, we discuss selection of parameters and evaluate the finite-sample performance by simulation. In Section 6, we re-analyze the dosage response data from Li & Barber (2015), illustrating the predictions of our theory in real data. |
| Researcher Affiliation | Academia | Lihua Lei EMAIL Department of Statistics, University of California, Berkeley William Fithian EMAIL Department of Statistics, University of California, Berkeley |
| Pseudocode | No | No pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | No | The paper states: "Where possible we have re-used the R code provided by Li & Barber (2015) at their website." However, it does not state that the authors are providing open-source code for their own proposed methodology (AS/SS). |
| Open Datasets | Yes | Li & Barber (2015) analyzed the performance of several ordered testing methods using the GEOquery data of Davis & Meltzer (2007). |
| Dataset Splits | No | The paper discusses data analysis and simulation but does not provide specific details on training, validation, or test dataset splits (e.g., percentages or sample counts). |
| Hardware Specification | No | The paper does not provide any specific hardware details (e.g., CPU, GPU, memory, or cloud instance types) used for running the experiments or simulations. |
| Software Dependencies | No | The paper mentions "R code" but does not specify any software dependencies with version numbers (e.g., specific R packages or versions). |
| Experiment Setup | Yes | We set q = 0.1, γ = 0.2, µ = 2, b = 3.65, in which case Π(0) = 0.75, Π(1) = 0.2. Each panel shows power for n = 100, 500, 1000, and 10, 000. For each setting we simulate 500 realizations... |