Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
From Counterfactuals to Trees: Competitive Analysis of Model Extraction Attacks
Authors: Awa Khouna, Julien Ferry, Thibaut Vidal
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments to validate our theoretical findings, presenting an averagecase and anytime performance analysis of TRA compared to state of the art reconstruction methods. These experiments not only confirm our theoretical results, but also provide practical insights into the effectiveness and limitations of our approach. |
| Researcher Affiliation | Academia | Awa Khouna Polytechnique Montréal EMAIL Julien Ferry Polytechnique Montréal EMAIL Thibaut Vidal Polytechnique Montréal EMAIL |
| Pseudocode | Yes | Algorithm 1 Tree Reconstruction Attack (TRA) |
| Open Source Code | Yes | The source code to reproduce all our experiments and figures is accessible at https://github.com/vidalt/Tree-Extractor, under an MIT license. |
| Open Datasets | Yes | We use five binary classification datasets, selected from related works on model extraction attacks [Aïvodji et al., 2020, Wang et al., 2022, Tramèr et al., 2016] and encompassing a variety of feature types, dimensionalities, and classification tasks, as summarized in Table 1. More precisely, we consider the COMPAS dataset [Angwin et al., 2016], as well as the Adult Income (Adult), Default of Credit Card Clients (Credit Card), German Credit and Student Performance (SPerformance) datasets from the UCI repository [Dua and Graff, 2017]. |
| Dataset Splits | Yes | Each dataset is partitioned into training, validation, and test sets with proportions of 60%, 20%, and 20%, respectively. |
| Hardware Specification | Yes | All experiments are run on a computing cluster with homogeneous nodes using Intel Platinum 8260 Cascade Lake @ 2.4GHz CPU. Each run uses four threads and up to 4GB of RAM each (multithreading is only used by the OCEAN oracle). |
| Software Dependencies | No | We train two types of tree-based target models implemented in the scikit-learn library [Pedregosa et al., 2011]: decision trees and random forests. |
| Experiment Setup | Yes | For decision trees, we experiment with varying max_depth parameters ranging from 4 to 10, as well as trees without maximum depth constraint (max_depth set to None). The random forests experiments focus on the COMPAS dataset, employing different numbers of trees (5, 25, 50, 75 and 100) to assess scalability and robustness. To prevent overfitting, we utilize the validation set for hyperparameter tuning and apply cost-complexity pruning where applicable. All the details of training procedures and hyperparameters configurations are discussed in Appendix C.1. |