Instance-Based Uncertainty Estimation for Gradient-Boosted Regression Trees
Authors: Jonathan Brophy, Daniel Lowd
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we find that IBUG achieves similar or better performance than the previous state-of-the-art across 22 benchmark regression datasets. We now compare IBUG s probabilistic and point predictions to NGBoost [22], PGBM [64], and CBU [44] on 21 benchmark regression datasets and one synthetic dataset. |
| Researcher Affiliation | Academia | Jonathan Brophy University of Oregon jbrophy@cs.uoregon.edu Daniel Lowd University of Oregon lowd@cs.uoregon.edu |
| Pseudocode | Yes | Algorithm 1 IBUG affinity computation. Algorithm 2 IBUG probabilistic prediction. Algorithm 3 IBUG accelerated tuning of k. |
| Open Source Code | Yes | Source code is available at https: //github.com/jjbrophy47/ibug. |
| Open Datasets | Yes | We now compare IBUG s probabilistic and point predictions to NGBoost [22], PGBM [64], and CBU [44] on 21 benchmark regression datasets and one synthetic dataset. Additional dataset details are in B.1. Links to all data sources as well as the code for IBUG and all experiments is available at https://github.com/jjbrophy47/ibug. |
| Dataset Splits | Yes | We use 10-fold cross-validation to create 10 90/10 train/test folds for each dataset. For each fold, the 90% training set is randomly split into an 80/20 train/validation set to tune any hyperparameters. |
| Hardware Specification | Yes | Experiments are run on publicly available datasets using an Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.6GHz with 60GB of RAM @ 2.4GHz. |
| Software Dependencies | No | The paper mentions software like Python, Cython, XGBoost, Light GBM, and Cat Boost, but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | We tune NGBoost the same way as in Duan et al. [22]. Since PGBM, CBU, and IBUG optimize a point prediction metric, we tune their hyperparameters similarly. We also tune variance calibration parameters γ and δ for each method ( 3.2). Exact hyperparameter values evaluated and selected are in B.2. |