Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Monotonic Calibrated Interpolated Look-Up Tables
Authors: Maya Gupta, Andrew Cotter, Jan Pfeifer, Konstantin Voevodski, Kevin Canini, Alexander Mangylov, Wojciech Moczydlowski, Alexander van Esbroeck
JMLR 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | A series of case studies in Section 10 experimentally explore the paper s proposals, and demonstrate that monotonic lattice regression achieves similar accuracy as a random forest, and that monotonicity is a common issue that arises in many different applications. The case studies demonstrate that for problems where the monotonicity assumption is warranted true, the proposed calibrated monotonic lattice regression produces similar accuracy to random forests. |
| Researcher Affiliation | Industry | Google 1600 Amphitheatre Pkwy Mountain View, CA 94301, USA |
| Pseudocode | Yes | Algorithm 1 Computes the multilinear interpolation weights and corresponding vertex indices for a unit lattice cell [0, 1]D and an x [0, 1]D. Let the lattice parameters be indexed such that sd = 2d is the difference in the indices of the parameters corresponding to any two vertices that are adjacent in the dth dimension, for example, for the 2 2 lattice, order the vertices [0 0], [1 0], [0 1], [1 1] and index the corresponding lattice parameters in that order. Calculate Multilinear Interpolation Weights And Parameter Indices(x) Algorithm 2 Computes the simplex interpolation weights and corresponding vertex indices for a unit lattice cell [0, 1]D and an x [0, 1]D. Let the lattice parameters be indexed such that sd = 2d is the difference in the indices of the parameters corresponding to any two vertices that are adjacent in the dth dimension, for example, for the 2 2 lattice, order the vertices [0 0], [1 0], [0 1], [1 1] and index the corresponding lattice parameters in that order. Calculate Simplex Interpolation Weights And Parameter Indices(x) |
| Open Source Code | No | The paper does not contain an explicit statement about open-sourcing the code for the described methodology or a link to a repository. |
| Open Datasets | No | The paper mentions several datasets (Business Matching, Ad Query Matching, Rendering Classifier, Fusing Pipelines, Video Ranking) used in case studies, including "anonymized data from You Tube", but does not provide any concrete access information (links, DOIs, specific citations to public repositories) for these datasets. They appear to be proprietary or internal datasets without public availability details. |
| Dataset Splits | Yes | The dataset was randomly split into mutually exclusive training, test, and validation sets of size 400 million, 25 million, and 25 million pairs, respectively. We used ten-fold cross-validation on each training set to choose hyperparameters |
| Hardware Specification | Yes | A single training loop through the 20,000 training samples took around five minutes on a Xeon-type Intel desktop using a single-threaded C++ implementation with sparse vectors |
| Software Dependencies | No | The paper mentions a "C++ implementation" but does not specify versions for compilers, libraries, or other software dependencies. |
| Experiment Setup | Yes | We used ten-fold cross-validation on each training set to choose hyperparameters, including: whether to use graph Laplacian regularization or torsion regularization, how much regularization (in powers of ten), whether to calibrate missing data or use a missing data vertex, the number of change-points if feature calibration was used from the choices: {2, 3, 5, 10, 20, 50}, and the number of vertices for each feature was started at 2 and increased by 1 as long as cross-validation accuracy increased. The step size was tuned using ten-fold cross-validation and choices were powers of 10; it was usually chosen to be one of {.01, .1, 1}. If calibration functions were used, a hyperparameter was used to scale the step size for the calibration function gradients compared to the lattice function gradients; this calibration step size scale was also chosen using ten-fold cross-validation and powers of 10, and was usually chosen to be one of {.01, .1, 1, 10}. |