Marich: A Query-efficient Distributionally Equivalent Model Extraction Attack
Authors: Pratik Karmakar, Debabrota Basu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Then, we evaluate MARICH on different text and image data sets, and different models, including CNNs and BERT. MARICH extracts models that achieve 60 95% of true model s accuracy and uses 1, 000 8, 500 queries from the publicly available datasets, which are different from the private training datasets. Models extracted by MARICH yield prediction distributions, which are 2 4 closer to the target s distribution in comparison to the existing active sampling-based attacks. The extracted models also lead to 84-96% accuracy under membership inference attacks. Experimental results validate that MARICH is query-efficient, and capable of performing task-accurate, high-fidelity, and informative model extraction. |
| Researcher Affiliation | Academia | Pratik Karmakar School of Computing, National University of Singapore CNRS@CREATE Ltd, 1 Create Way, Singapore pratik.karmakar@u.nus.edu Debabrota Basu Équipe Scool, Univ. Lille, Inria, CNRS, Centrale Lille UMR 9189CRISt AL, F-59000 Lille, France debabrota.basu@inria.fr |
| Pseudocode | Yes | Algorithm 1 MARICH Input: Target model: f T , Query dataset: DQ, #Classes: k Parameter: #initial samples: n0, Training epochs: Emax, #Batches of queries: T, Query budget: B, Subsampling ratios: γ1, γ2 (0, 1] Output: Extracted model f E |
| Open Source Code | Yes | Code is available at: https://github.com/debabrota-basu/marich |
| Open Datasets | Yes | For model extraction, we use EMNIST letters dataset [CATv S17], CIFAR10 [KH+09], Image Net [DDS+09], and AGNews [ZZL15], as publicly-available, mismatched query datasets DQ. |
| Dataset Splits | Yes | Test accuracy of the extracted model and its comparison with the test accuracy of the target model on a subset of the private training dataset, which was used by neither of these models, is the most common performance metric used to evaluate the goodness of the attack algorithm. We compute test accuracies of the target models, and models extracted by MARICH and other active sampling algorithms in six experimental setups. For model extraction, we use EMNIST letters dataset [CATv S17], CIFAR10 [KH+09], Image Net [DDS+09], and AGNews [ZZL15], as publicly-available, mismatched query datasets DQ. |
| Hardware Specification | Yes | We implement a prototype of MARICH using Python 3.9 and Py Torch 1.12, and run on a NVIDIA Ge Force RTX 3090 24 GB GPU. |
| Software Dependencies | Yes | We implement a prototype of MARICH using Python 3.9 and Py Torch 1.12, and run on a NVIDIA Ge Force RTX 3090 24 GB GPU. |
| Experiment Setup | Yes | Table 7: Hyperparameters for different datasets and target models. Member Dataset Target Model Attack Model Attack Dataset Budget Initial points γ1 γ2 Rounds Epochs/Round Learning Rate |