Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Launch and Iterate: Reducing Prediction Churn
Authors: Mahdi Milani Fard, Quentin Cormier, Kevin Canini, Maya Gupta
NeurIPS 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on benchmark datasets for different classification algorithms demonstrate the method and the resulting reduction in churn. |
| Researcher Affiliation | Collaboration | Q. Cormier ENS Lyon 15 parvis René Descartes Lyon, France EMAIL M. Milani Fard, K. Canini, M. R. Gupta Google Inc. 1600 Amphitheatre Parkway Mountain View, CA 94043 EMAIL |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any information about open-source code being made available for the described methodology. |
| Open Datasets | Yes | This section demonstrates the churn reduction effect of the RCP operator for three UCI benchmark datasets (see Table 2) with three regression algorithms: ridge regression, random forest regression, and support vector machine regression with RBF kernel, all implemented in Scikit-Learn [12]. [...] Nomao [13] News Popularity [14] Twitter Buzz [15] |
| Dataset Splits | Yes | We randomly split each dataset into three fixed parts: a training set, a validation set on which we optimized the hyper-parameters for all algorithms, and a testing set. [...] Table 2: Validation set 1000 samples (for each dataset) |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU, GPU models, or memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions 'all implemented in Scikit-Learn [12]' but does not specify a version number for Scikit-Learn or any other software dependencies. |
| Experiment Setup | Yes | We used fixed values of α = 0.5 and ϵ = 0.5 for all the experiments in Table 3 [...] The dataset perturbation sub-samples 80% of the examples in TA and randomly drops 3-7 features. We run 40 independent chains to measure the variability, and report the average outcome and standard deviation. [...] See the supplementary material for more experimental details. |