Measuring Data Reconstruction Defenses in Collaborative Inference Systems
Authors: Mengda Yang, Ziang Li, Juan Wang, Hongxin Hu, Ao Ren, Xiaoyang Xu, Wenzhe Yi
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that SFD can break through defense mechanisms in model partitioning scenarios, demonstrating the inadequacy of existing defense mechanisms as a privacy-preserving technique against reconstruction attacks. We conduct a comprehensive measurement of the robustness of various state-of-the-art defenses leveraging our proposed technique. Our experiments show that SFD can achieve 74.2% performance improvement compared with the baseline attack, making the existing defenses significantly less effective. |
| Researcher Affiliation | Academia | 1Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University 2Department of Computer Science and Engineering, University at Buffalo, SUNY 3College of Computer Science, Chongqing University |
| Pseudocode | No | The paper describes its methods in narrative text and mathematical formulations but does not include formal pseudocode or algorithm blocks. |
| Open Source Code | No | In the ethics checklist, the authors state for question 3(a) 'Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?' that the answer is [No]. |
| Open Datasets | Yes | To perform experimental evaluation, we use three standard benchmark image recognition datasets: MNIST [Le Cun, 1998], CIFAR10 [Krizhevsky et al., 2009] and Celeb A [Liu et al., 2015]. |
| Dataset Splits | Yes | In most of our experiments, we split the dataset into three parts, target models private dataset Dp, adversary s shadow dataset Ds and the test set Dt with the rest. We detail each dataset in the following: MNIST contains 70,000 grey-scale images. We set Dp, Ds and Dt to 50,000, 10,000 and 10,000 respectively; CIFAR10 consists of 60,000 color images. We separate Dp, Ds and Dt to 40,000, 10,000, and 10,000 respectively; Celeb A is a dataset composed of 202,599 images of celebrities. ... The Dp, Ds and Dt are set to 101,299, 81,040 and 20,260 accordingly. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, or cloud platforms) used to run the experiments. |
| Software Dependencies | No | The paper does not specify software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | Table 1 gives details of the experimental configurations. To quantify the privacy risks of reconstruction attacks, we follow the setting in most of the previous work [He et al., 2019; Singh et al., 2021]. We adopt two metrics, Mean Squared Error (MSE) and Structural Similarity Index (SSIM) [Wang et al., 2004]. ... Defense hyper-parameters are set λ = 5.0 for adversarial learning, b = 10.0 for noise mask, and r = 0.9 for dropout defense. |