Boosting with Tempered Exponential Measures
Authors: Richard Nock, Ehsan Amid, Manfred Warmuth
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We have performed experiments on a testbed of 10 UCI domains, whose details are given in APP. (Section A3). Experiments were carried out using a 10-fold stratified cross-validation procedure. To compare t-ADABOOST with ADABOOST, we ran t-ADABOOST with a first range of values of t P t0.0, 0.2, 0.4, 0.6, 0.8, 0.9u. |
| Researcher Affiliation | Industry | Richard Nock Google Research richardnock@google.com Ehsan Amid Google Deep Mind eamid@google.com Manfred K. Warmuth Google Research manfred@google.com |
| Pseudocode | Yes | Algorithm 1 t-ADABOOSTpt, S, Jq Input: t P r0, 1s, training sample S, #iterations J; Output: classifiers HJ, Hp1{1 tq J (see (9)); Step 1 : initialize tempered weights: q1 p1{mt q 1 p P mq; Step 2 : for j 1, 2, ..., J Step 2.1 : get weak classifier hj Ð weak_learnerpqj, Sq; Step 2.2 : choose weight update coefficient µj P R; Step 2.3 : @i P rms, for uji . yihjpxiq, update the tempered weights as qpj 1qi qji bt exptp µjujiq Ztj , where Ztj qj bt exptp µjujq 1{t . (8) Step 2.4 : choose leveraging coefficient αj P R; |
| Open Source Code | No | The paper does not provide a statement or link indicating that the source code for their methodology is publicly available. |
| Open Datasets | Yes | We have performed experiments on a testbed of 10 UCI domains, whose details are given in APP. (Section A3). Table A3: Public domains considered in our experiments (m total number of examples, d total number of example s features, including class), ordered in increasing m ˆ d (see text). |
| Dataset Splits | Yes | Experiments were carried out using a 10-fold stratified cross-validation procedure. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments (e.g., CPU, GPU models, memory). |
| Software Dependencies | No | The paper states that the implementation is "programmed in Java" but does not specify any version numbers for Java or other software libraries/dependencies. |
| Experiment Setup | Yes | All boosting models were trained for a number of J 20 decision trees (The appendix provides experiments on training bigger sets). Each decision tree is induced using the tempered loss with the corresponding value of t (see Theorem 4) following the classical top-down template, which consists in growing the current heaviest leaf in the tree and picking the best split for the leaf chosen. We implemented t-ADABOOST exactly as in Section 5, including computing leveraging coefficients as suggested. Thus, we do not scale models. More details are provided in the appendix. In our implementation of decision-tree induction, when the number of possible splits exceeds a fixed number S (currently, 2 000), we pick the best split in a subset of S splits picked at random. |