A Neural Tangent Kernel Perspective of Infinite Tree Ensembles
Authors: Ryuichi Kanoh, Mahito Sugiyama
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comparison to the NTK induced by the MLP. We investigate the generalization performance of infinite tree ensembles by kernel regression with the TNTK on 90 real-world datasets. |
| Researcher Affiliation | Academia | 1National Institute of Informatics 2The Graduate University for Advanced Studies, SOKENDAI {kanoh, mahito}@nii.ac.jp |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | Yes | For the numerical experiments and figures, we share our reproducible source code in the supplementary material. |
| Open Datasets | Yes | We use the UCI datasets (Dua & Graff, 2017) preprocessed by Fernández-Delgado et al. (2014), which are publicly available at http://persoal.citius.usc.es/manuel.fernandez. delgado/papers/jmlr/data.tar.gz. |
| Dataset Splits | Yes | We report 4-fold cross-validation performance with random data splitting. To tune parameters, all available training samples are randomly split into one training and one validation set, while imposing that each class has the same number of training and validation samples. |
| Hardware Specification | Yes | We used Ubuntu Linux (version: 4.15.0-117-generic) and ran all experiments on 2.20 GHz Intel Xeon E5-2698 CPU and 252 GB of memory. |
| Software Dependencies | No | The paper mentions using 'Ubuntu Linux' and 'scikit-learn implementation' but does not provide specific version numbers for software dependencies beyond the OS kernel version. |
| Experiment Setup | Yes | We performed kernel regression using the limiting TNTK defined in Equation (7) with varying the tree depth (d) and the scaling (α) of the decision function. [...] regularization strength is set to be 1.0e-8, a very small constant. |