Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Conformalized Multiple Testing after Data-dependent Selection
Authors: Xiaoning Wang, Yuyang Huo, Liuhua Peng, Changliang Zou
NeurIPS 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Furthermore, we conduct numerical studies to showcase the effectiveness and validity of our procedures across various scenarios. |
| Researcher Affiliation | Academia | 1School of Statistics and Data Sciences, LPMC, KLMDASR and LEBPS, Nankai University, Tianjin, China 2School of Mathematics and Statistics, The University of Melbourne, Melbourne, Australia |
| Pseudocode | Yes | Algorithm 1 Selective conformal p-value with BH procedure (SCPV) |
| Open Source Code | Yes | We provide the data and code in the supplemental materials, including instructions in the zip file. |
| Open Datasets | Yes | Abalone[37]: contains easily obtainable measurements of abalone. The task is to predict the age of abalone from physical measurements. |
| Dataset Splits | No | The paper mentions a 'labeled/holdout data set Dc' and an 'unlabelled/test set Du', and their sizes, but does not explicitly specify a separate validation dataset split with proportions. |
| Hardware Specification | Yes | All the experiments were conducted on AMD Ryzen 7 5800H with Radeon Graphics processor with 16 Gb memory at a Lenovo personal computer and the R platform with version 4.2.1. |
| Software Dependencies | Yes | All the experiments were conducted on AMD Ryzen 7 5800H with Radeon Graphics processor with 16 Gb memory at a Lenovo personal computer and the R platform with version 4.2.1. |
| Experiment Setup | Yes | We fix the labeled data size n = 1, 200 and the unlabeled data size m = 1, 200. We fit the regression models หยต( ) on an additional labeled set with size 1, 200 using the random forest algorithm, implemented by R package random Forest with default parameters. |