Better by default: Strong pre-tuned MLPs and boosted trees on tabular data
Authors: David Holzmüller, Leo Grinsztajn, Ingo Steinwart
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our benchmark results on medium-to-large tabular datasets (1K 500K samples) show that Real MLP offers a favorable time-accuracy tradeoff compared to other neural baselines and is competitive with GBDTs in terms of benchmark scores. |
| Researcher Affiliation | Academia | David Holzmüller SIERRA Team, Inria Paris Ecole Normale Superieure PSL University Léo Grinsztajn SODA Team, Inria Saclay Ingo Steinwart University of Stuttgart Faculty of Mathematics and Physics Institute for Stochastics and Applications |
| Pseudocode | No | No pseudocode or algorithm blocks are present in the paper. |
| Open Source Code | Yes | The code for our benchmarks, including scikit-learn interfaces for the models, is available at https://github.com/dholzmueller/pytabkit Our code and data are archived at https://doi.org/10.18419/darus-4555. |
| Open Datasets | Yes | We evaluate our methods on the benchmark by Grinsztajn et al. [18] as well as datasets from the Auto ML benchmark [13] and the Open ML-CTR23 regression benchmark [12]. |
| Dataset Splits | Yes | To this end, we evaluate a method on Nsplits = 10 random training-validation-test splits (60%-20%-20%) on each dataset. |
| Hardware Specification | Yes | We run all methods on a single compute node with a 32-core AMD Ryzen Threadripper Pro 3975 WX CPU, using 32 threads for GBDTs and the Py Torch default settings for NNs. |
| Software Dependencies | No | Our implementation uses various libraries, out of which we would like to particularly acknowledge Py Torch [47], Scikit-learn [48], Ray [46], XGBoost [9], Light GBM [31], and Cat Boost [51]. |
| Experiment Setup | Yes | The detailed hyperparameters can be found in Table A.1. |