Versatile Verification of Tree Ensembles
Authors: Laurens Devos, Wannes Meert, Jesse Davis
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally show that our method produces state-of-the-art robustness estimates, especially when executed with strict time constraints. This is exceedingly important when checking the robustness of large datasets. Additionally, we show that VERITAS enables tackling more real-world verification scenarios. |
| Researcher Affiliation | Academia | 1Department of Computer Science, KU Leuven, Leuven, Belgium. Correspondence to: Laurens Devos <laurens.devos@kuleuven.be>. |
| Pseudocode | No | The paper describes the algorithm's steps in paragraph form and through mathematical equations but does not present a formal pseudocode or algorithm block. |
| Open Source Code | Yes | VERITAS is available as an open-source package.1 1https://github.com/laudv/veritas |
| Open Datasets | Yes | We compare on seven commonly used datasets for checking robustness (e.g., (Chen et al., 2019b)). All models were trained using XGBoost (Chen & Guestrin, 2016) using the same number of trees and tree depth as reported in (Chen et al., 2019b). We performed hyperparameter optimization to tune the learning rate. All details of the datasets and parameters are summarized in the supplementary materials. All datasets except MNIST and Fashion-MNIST were minmax-normalized. |
| Dataset Splits | No | The paper mentions training data and test data but does not explicitly describe the use of a separate validation set for hyperparameter tuning or early stopping, nor does it specify train/validation/test splits. |
| Hardware Specification | Yes | All experiments ran on an Intel(R) Xeon(R) CPU E3-1225 with 32Gi B of memory. VERITAS s memory usage was restricted to 1Gi B, and never used more than 150Mi B. MERGE s memory limit was increased to 8Gi B as it often failed to run with 4Gi B of memory. |
| Software Dependencies | Yes | We used our own implementation of the MILP approach with Gurobi 9.1.1 (Gurobi Optimization, 2021) as the solver. |
| Experiment Setup | Yes | All models were trained using XGBoost (Chen & Guestrin, 2016) using the same number of trees and tree depth as reported in (Chen et al., 2019b). We performed hyperparameter optimization to tune the learning rate. All details of the datasets and parameters are summarized in the supplementary materials. |