Circa: Stochastic ReLUs for Private Deep Learning
Authors: Zahra Ghodsi, Nandan Kumar Jha, Brandon Reagen, Siddharth Garg
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section we evaluate Circa and validate our error model. We show that our optimizations have minimal effect on network accuracy and show the runtime and storage benefits of Circa. We perform experiments on Res Net18 [11], Res Net32 [11] and VGG16 [12]. We train these networks on CIFAR-10/100 [13] and Tiny Image Net [14] datasets in plaintext (Circa is not involved). The baseline accuracy of the models in PI is reported using an integer model with network values in a prime field. To obtain an integer model, we scale and quantize model parameters and input to 15 bits (as in Delphi), and pick a 31 bit prime field (p = 2138816513) to ensure that multiplication of two 15-bit values does not exceed the field. The baseline accuracy of our integer models is reported in Table 1. |
| Researcher Affiliation | Academia | Zahra Ghodsi1, Nandan Kumar Jha2, Brandon Reagen2, Siddharth Garg2 1University of California San Diego, 2New York University zghodsi@ucsd.edu, {nj2049, bjr5, sg175}@nyu.edu |
| Pseudocode | No | The paper provides mathematical equations (e.g., Eq. 1, 2, 3) and block diagrams (Figure 2) describing the proposed functions and their components, but it does not include formal pseudocode or algorithm blocks. |
| Open Source Code | No | The paper states: 'We use the SEAL library [15] for HE, and fancy-garbling library [16] for GC.' However, it does not provide any explicit statement or link indicating that the authors' own implementation code for Circa is open-source or publicly available. |
| Open Datasets | Yes | We train these networks on CIFAR-10/100 [13] and Tiny Image Net [14] datasets in plaintext (Circa is not involved). |
| Dataset Splits | No | CIFAR-10/100 (C10/100) datatset has 50k training and 10k test images (size 32 32) separated into 10/100 output classes. Tiny Image Net (Tiny) consist of 200 output classes with 500 training and 50 test samples (size 64 64) per class. The paper specifies training and test set sizes but does not explicitly describe a validation split. |
| Hardware Specification | Yes | We benchmark PI runtime on an Intel i9-10900X CPU running at 3.70GHz with 64GB of memory. |
| Software Dependencies | No | The paper states: 'We use the SEAL library [15] for HE, and fancy-garbling library [16] for GC.' However, specific version numbers for these software dependencies are not provided. |
| Experiment Setup | Yes | The training procedure uses stochastic gradient descent with step learning optimizer and 0.1 initial learning rate, 128 batch size, 0.0001 weight deacy, 0.9 momentum, and milestones at 100th and 150th epochs for 200 epochs. |