Gradient Boosting Performs Gaussian Process Inference
Authors: Aleksei Ustimenko, Artem Beliakov, Liudmila Prokhorenkova
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments confirm that the proposed sampler from the Gaussian process posterior outperforms the previous approaches (Malinin et al., 2021) and gives better knowledge uncertainty estimates and improved out-of-domain detection. |
| Researcher Affiliation | Collaboration | Aleksei Ustimenko Share Chat research@aleksei.uk Artem Beliakov Yandex Research, HSE University belyakov.arteom2015@gmail.com Liudmila Prokhorenkova Yandex Research ostroumova-la@yandex.com |
| Pseudocode | Yes | Algorithm 1 Sample Tree(r; m, n, β), Algorithm 2 Train GBDT(z; ϵ, T, m, n, β, λ), Algorithm 3 Sample Prior(T, m, n), Algorithm 4 Sample Posterior(z; ϵ, T1, T0, m, n, β, σ, δ) |
| Open Source Code | Yes | The code of our experiments can be found on Git Hub.7 https://github.com/Take Over/Gradient-Boosting-Performs-Gaussian-Process-Inference |
| Open Datasets | Yes | For the experiments, we use several standard regression datasets (Gal & Ghahramani, 2016). |
| Dataset Splits | Yes | As a hyperparameter (that is estimated on the validation set), we consider β {10 2, 10 1, 1}., Here we perform cross-validation to estimate statistical significance with paired t-test and highlight the approaches that are insignificantly different from the best one (p-value > 0.05). |
| Hardware Specification | No | The paper does not provide specific hardware details such as exact GPU/CPU models, processor types, or memory amounts used for running the experiments. It only mentions 'Implementation details' in Appendix H, which does not include hardware specifications. |
| Software Dependencies | No | The paper mentions using 'the standard Cat Boost library' but does not specify a version number for it or any other software dependencies, which is required for a reproducible description. |
| Experiment Setup | Yes | For KGB, we fix ϵ = 0.3, T0 = 100, T1 = 900, σ = 10 2, δ = 10 4 β = 0.1, m = 4, n = 64, and sampled 100 KGB models., As a hyperparameter (that is estimated on the validation set), we consider β {10 2, 10 1, 1}., Finally, we set l2 leaf reg value to 0, as SGLB does. |