Generalized Linear Rule Models
Authors: Dennis Wei, Sanjeeb Dash, Tian Gao, Oktay Gunluk
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments are presented involving the two most common cases of GLMs, logistic and linear regression. The proposed methods are seen to yield better performance-complexity trade-offs than existing rule ensemble algorithms. At the performance-maximizing end of the trade-off, the methods are competitive with less interpretable benchmark models such as tree ensembles and nonlinear support vector machines (SVM). The trade-off curves also suggest that substantially simpler models are often available at a relatively small cost in performance. |
| Researcher Affiliation | Industry | 1IBM Research, Yorktown Heights, NY, USA. |
| Pseudocode | No | The paper describes algorithms in prose but does not provide structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper mentions a Python implementation for Rule Fit, a comparison method, at 'https://github.com/christophM/rulefit', but does not provide a link or statement for the open-source code of their own proposed methods. |
| Open Datasets | Yes | For classification, we used the same 16 datasets considered in (Dash et al., 2018), which also appeared in other recent works on rule-based models (Su et al., 2016; Wang et al., 2017). One of these datasets comes from the recent FICO Explainable Machine Learning Challenge (FICO, 2018). For regression, we experimented with an additional 8 datasets, 7 of which are drawn from previous works on rule ensembles (Friedman & Popescu, 2008; Dembczy nski et al., 2010) and the UCI repository (Dua & Karra Taniskidou, 2017). The last dataset comes from the Medical Expenditure Panel Survey (MEPS) (Agency for Healthcare Research and Quality, 2018) of the US Department of Health and Human Services, specifically panel 19 from the year 2015. |
| Dataset Splits | Yes | 10-fold cross-validation (CV) is used to estimate all test performance metrics. In a second experiment, we aim to maximize performance (minimize Brier score or maximize accuracy) by performing nested CV on the training set to select λ0 and applying the resulting model to the test set. |
| Hardware Specification | No | The paper mentions using "Cplex callable library (version 12.7.1)" for solving an integer program, but it does not specify any hardware components (e.g., CPU, GPU, memory) used for running the experiments. |
| Software Dependencies | Yes | We have implemented our column generation procedure in Java using Lib Linear (Fan et al., 2008) to solve the regularized logistic regression problem (4) and using Cplex callable library (version 12.7.1) to solve the integer program (9) for column generation. |
| Experiment Setup | Yes | We assume that the regularization parameter λk is an affine function of the degree of the conjunction, λk = λ0 + λ1 Pj zj with λ0, λ1 0 (other affine functions of zj are possible). We set λ1/λ0 = 0.2. For GBT, the maximum tree depth was tuned and the number of trees was also determined via a stopping criterion on a validation set, up to a maximum of 500 trees. For SVM, the regularization parameter C and kernel width γ were tuned and Platt scaling (Platt, 1999) was used to calibrate the output scores as probabilities. |