Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data
Authors: Timothy J Castiglia, Anirban Das, Shiqiang Wang, Stacy Patterson
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we experimentally show compression can reduce communication by over 90% without a significant decrease in accuracy over VFL without compression. |
| Researcher Affiliation | Collaboration | 1Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY, USA 2IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA. |
| Pseudocode | Yes | Algorithm 1 Compressed Vertical Federated Learning; Algorithm 2 Compressed Vertical Federated Learning for Q = 1 |
| Open Source Code | Yes | Our code for training with the MIMIC-III dataset can be found in in the folder titled mimic3 . This is an extension of the MIMIC-III benchmarks repo found at: github.com/Yereva NN/mimic3-benchmarks. Our code for learning on the Model Net10 dataset is in the folder MVCNN Pytorch and is an extension of the MVCNNPy Torch repo: github.com/RBirkeland/MVCNN-Py Torch. In the same folder, MVCNN Pytorch , we include our code for running CIFAR-10. The file of interest is quant cifar.py which trains C-VFL with CIFAR-10. Our code for learning on the Image Net dataset is in the folder Image Net CVFL and is a modification on the moco align uniform repo: https://github.com/Ssn L/moco align uniform. |
| Open Datasets | Yes | We run experiments on three datasets: the MIMIC-III dataset (Johnson et al., 2016), the Model Net10 dataset (Wu et al., 2015), and the CIFAR-10 dataset (Krizhevsky et al., 2009). ... MIMIC-III: The MIMIC-III dataset can be found at: mimic.physionet.org. ... Model Net10: Details on the Model Net10 dataset can be found at: modelnet.cs.princeton.edu/. ... CIFAR-10: More information on the CIFAR-10 dataset can be found at: cs.toronto.edu/ kriz/cifar.html. |
| Dataset Splits | No | For in-hospital mortality, there are 14,681 training samples, and 3,236 test samples. The paper specifies train and test splits for MIMIC-III but does not explicitly mention a separate validation split or specific percentages/counts for one for any of the datasets used in the main text. While common datasets often have standard validation splits, the paper does not specify how validation was handled for its experiments. |
| Hardware Specification | Yes | For our experiments, we used an internal cluster of 40 compute nodes running Cent OS 7 each with 2 20-core 2.5 GHz Intel Xeon Gold 6248 CPUs, 8 NVIDIA Tesla V100 GPUs with 32 GB HBM, and 768 GB of RAM. |
| Software Dependencies | No | Our code has updated their existing code to Tensor Flow 2. ... Our code for learning on the Model Net10 dataset is in the folder MVCNN Pytorch. ... We use the version of CIFAR-10 downloaded through the torchvision library. The paper mentions software like TensorFlow 2 and PyTorch, but it does not provide specific version numbers for these libraries or other critical software dependencies. |
| Experiment Setup | Yes | MIMIC-III: ...We use a fixed step size of 0.01, a batch size of 1000, and train for 1000 epochs. CIFAR-10: ...We use a fixed step size of 0.0001 and a batch size of 100, and train for 200 epochs. Model Net10: ...We use a fixed step size of 0.001 and a batch size of 64, and train for 100 epochs. Image Net: ...We use a variable step size, that starts at 0.001, and drops to 0.0001 after 50 epochs. We use a batch size of 256 and train for 100 epochs. |