Alternating optimization of decision trees, with application to learning sparse oblique trees
Authors: Miguel A. Carreira-Perpinan, Pooya Tavallali
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show it consistently outperforms various other algorithms while being highly scalable to large datasets and trees. ... We constructed paths using initial CART trees of depths 4 to 12 (both axis-aligned and oblique) on the MNIST dataset, splitting its training set of 60k points into 48k training and 12k validation (to determine an optimal C or depth), and reporting generalization error on its 10k test points. |
| Researcher Affiliation | Academia | Miguel A. Carreira-Perpi n an Dept. EECS, University of California, Merced mcarreira-perpinan@ucmerced.edu Pooya Tavallali Dept. EECS, University of California, Merced ptavallali@ucmerced.edu |
| Pseudocode | Yes | TAO pseudocode appears in the supplementary material. |
| Open Source Code | No | The paper states, "The node optimization uses an ℓ1regularized linear SVM with slack hyperparameter C 0 ... implemented with LIBLINEAR [15]. The rest of our code is in Matlab." While it mentions using LIBLINEAR and Matlab, it does not explicitly state that the authors' own implementation code for TAO is open-source or publicly available. |
| Open Datasets | Yes | We constructed paths using initial CART trees of depths 4 to 12 (both axis-aligned and oblique) on the MNIST dataset... [27] MNIST. The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist. |
| Dataset Splits | Yes | splitting its training set of 60k points into 48k training and 12k validation (to determine an optimal C or depth), and reporting generalization error on its 10k test points. |
| Hardware Specification | Yes | ran on a laptop with 2 core i5 CPUs and 12GB RAM |
| Software Dependencies | No | The node optimization uses an ℓ1regularized linear SVM with slack hyperparameter C 0 ... implemented with LIBLINEAR [15]. The rest of our code is in Matlab. The paper mentions LIBLINEAR and Matlab but does not provide specific version numbers for either. |
| Experiment Setup | Yes | The node optimization uses an ℓ1regularized linear SVM with slack hyperparameter C 0 (so the TAO sparsity hyperparameter in section 3 is λ = 1/C), implemented with LIBLINEAR [15]. ... We stop TAO when the training misclassification loss decreases but by less than 0.5%, or the number of iterations (passes over all nodes) reaches 14 (in practice TAO stops after around 7 iterations). ... We constructed paths using initial CART trees of depths 4 to 12 (both axis-aligned and oblique) on the MNIST dataset... |