Online Gradient Boosting
Authors: Alina Beygelzimer, Elad Hazan, Satyen Kale, Haipeng Luo
NeurIPS 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we conduct some proof-of-concept experiments which show that our online boosting algorithms do obtain performance improvements over di erent classes of base learners. All experiments were done on a collection of 14 publically available regression and classification datasets (described in Section D in the supplementary material) using squared loss. The following table reports the average and the median, over the datasets, relative improvement in squared loss over the respective base learner. |
| Researcher Affiliation | Collaboration | Alina Beygelzimer Yahoo Labs New York, NY 10036 beygel@yahoo-inc.com Elad Hazan Princeton University Princeton, NJ 08540 ehazan@cs.princeton.edu Satyen Kale Yahoo Labs New York, NY 10036 satyen@yahoo-inc.com Haipeng Luo Princeton University Princeton, NJ 08540 haipengl@cs.princeton.edu |
| Pseudocode | Yes | Algorithm 1 Online Gradient Boosting for span(F) Algorithm 2 Online Gradient Boosting for CH(F) |
| Open Source Code | No | Is it possible to boost in an online fashion in practice with real base learners? To study this question, we implemented and evaluated Algorithms 1 and 2 within the Vowpal Wabbit (VW) open source machine learning system [23]. [23] VW. URL https://github.com/John Langford/vowpal_wabbit/. The paper states that the algorithms were implemented *within* Vowpal Wabbit (an existing open-source system), but it does not state that the code for *their specific implementations* of Algorithms 1 and 2 is open-source or provided. |
| Open Datasets | Yes | All experiments were done on a collection of 14 publically available regression and classification datasets (described in Section D in the supplementary material) using squared loss. |
| Dataset Splits | Yes | Parameters were tuned based on progressive validation loss on half of the dataset; reported is propressive validation loss on the remaining half. Progressive validation is a standard online validation technique, where each training example is used for testing before it is used for updating the model [3]. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU/CPU models, memory, or cloud instance types used for running the experiments. It only states that experiments were conducted. |
| Software Dependencies | No | The paper states that the algorithms were 'implemented and evaluated [...] within the Vowpal Wabbit (VW) open source machine learning system [23]', but it does not specify a version number for Vowpal Wabbit or any other software dependencies. |
| Experiment Setup | Yes | The only parameters tuned were the learning rate and the number of weak learners, as well as the step size parameter for Algorithm 1. |