Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
A Model-free Closeness-of-influence Test for Features in Supervised Learning
Authors: Mohammad Mehrabi, Ryan A. Rossi
ICML 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our findings through extensive numerical simulations, specifically we adopt the datamodel framework (Ilyas, et al., 2022) for CIFAR-10 dataset to identify pairs of training samples with different influence on the trained model via optional black box training mechanisms. |
| Researcher Affiliation | Collaboration | 1Department of Data Sciences and Operations, University of Southern California, Los Angeles, USA 2Adobe, San Jose, USA. |
| Pseudocode | Yes | Algorithm 1 Test statistic for hypothesis testing 1 |
| Open Source Code | No | The paper provides a link to |
| Open Datasets | Yes | We consider the CIFAR-10 dataset (Krizhevsky et al., 2009) |
| Dataset Splits | No | The paper mentions training and testing samples but does not explicitly describe a validation split for model training or evaluation. It mentions |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory, or cloud instance types) used for running the experiments. It vaguely mentions |
| Software Dependencies | No | The paper mentions using |
| Experiment Setup | Yes | We run our method with the score function T(x, y) = |y bθTx| with bθ N(0, Id). The estimate bθ is fixed across all 45 tests. We suppose that we have access to 1000 data points, and we consider three different significance levels α = 0.1, 0.15, and 0.2. |