Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Hidden Technical Debt in Machine Learning Systems
Authors: D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-François Crespo, Dan Dennison
NeurIPS 2015 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | This paper does not offer novel ML algorithms, but instead seeks to increase the community s awareness of the difficult tradeoffs that must be considered in practice over the long term. We focus on system-level interactions and interfaces as an area where ML technical debt may rapidly accumulate. |
| Researcher Affiliation | Industry | D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips EMAIL Google, Inc. Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Franc ois Crespo, Dan Dennison EMAIL Google, Inc. |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | No | The paper is a conceptual discussion of technical debt in ML systems and does not present a specific methodology or tool for which source code would be released. It does not mention any open-source code for its own described work. |
| Open Datasets | No | The paper is a conceptual discussion of technical debt in ML systems and does not describe experiments that use a specific dataset for training or evaluation. It references general practices and systems but not specific public datasets used in its own research. |
| Dataset Splits | No | The paper is a conceptual work and does not conduct experiments, therefore, it does not specify any training/validation/test dataset splits. |
| Hardware Specification | No | The paper is a conceptual discussion of technical debt in ML systems and does not describe any experiments that would require specific hardware specifications. |
| Software Dependencies | No | The paper is a conceptual discussion and does not describe any specific software implementation details or experiments that would require listing software dependencies with version numbers. |
| Experiment Setup | No | The paper is a conceptual discussion and does not describe any experiments or specific model training, thus it does not provide details about experimental setup or hyperparameters. |