Batchwise Patching of Classifiers
Authors: Sebastian Kauschke, Johannes Fürnkranz
AAAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally evaluate our approach, and show that it meets the original design goals. Moreover, we compare our approach to existing methods from the domain of ensemble stream classification in both concept drift and transfer learning situations. |
| Researcher Affiliation | Academia | Sebastian Kauschke,1,2 Johannes F urnkranz1 1 Knowledge Engineering Group, 2 Telecooperation Group TU Darmstadt, Germany {kauschke, fuernkranz}@ke.tu-darmstadt.de |
| Pseudocode | No | The paper describes the approach verbally and mathematically but does not include any pseudocode or algorithm blocks. |
| Open Source Code | Yes | The entire patching framework is implemented in MOA and publicly available on Git Hub 5. https://github.com/Shademan/Patching/releases/tag/AAAI2018 |
| Open Datasets | Yes | The first dataset is the MNIST dataset of handwritten digits. The second dataset that we will use in our evaluation is 20 Newsgroups. The SEA concepts dataset is a dataset with abrupt concept drift (Street and Kim 2001). The last dataset we use is based on a rotating hyperplane in a d-dimensional space as proposed in (Hulten, Spencer, and Domingos 2001). |
| Dataset Splits | No | We use a chunk/batch-based evaluation method to retrieve the performance of the classifier every m instances, for which we split each dataset into a certain amount of batches (Tab. 1). This way, we can use the whole dataset for training and evaluation without having to rely on a separate hold-out set. |
| Hardware Specification | No | We gratefully acknowledge the use of the Lichtenberg high performance computer of the TU Darmstadt for our experiments. |
| Software Dependencies | No | We use the Massive Online Analysis (MOA) framework1 (Bifet et al. 2010) and the WEKA toolkit2 (Witten and Frank 2005) for machine learning, thereby simulating a real-world scenario. |
| Experiment Setup | Yes | In our experiments, we primarily use random forests (RF) with 100 random trees... The base classifier for Patching is trained as a random forest with 100 random trees. For AWE, we set the ensemble size to 5 random forests, each consisting of 100 random trees. AUE is configured similarly, with a maximum of 500 random trees. Oza Boost is allowed to use 500 random Hoeffding Trees... In our experiments we used a compromise value of n = 8 which yielded overall adequate results in preliminary runs. |