Flow: Per-instance Personalized Federated Learning
Authors: Kunjal Panchal, Sunav Choudhary, Nisarg Parikh, Lijun Zhang, Hui Guan
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide theoretical analysis on the convergence of Flow and empirically demonstrate the superiority of Flow in improving clients accuracy compared to state-of-the-art personalization approaches on both vision and language-based tasks. We empirically evaluate the superiority of Flow in both generalization and personalized accuracy on various vision and language tasks in cross-device FL settings. |
| Researcher Affiliation | Collaboration | Kunjal Panchal College of Information and Computer Sciences University of Massachusetts, Amherst Amherst, MA 01003 kpanchal@umass.edu Sunav Choudhary Adobe Research Bangalore, India schoudha@adobe.com Nisarg Parikh Khoury College of Computer Sciences Northeastern University Boston, MA 02115 parikh.nis@northeastern.edu Lijun Zhang, Hui Guan College of Information and Computer Sciences University of Massachusetts, Amherst Amherst, MA 01003 {lijunzhang, huiguan}@umass.edu |
| Pseudocode | Yes | Algorithm 1: Pseudocode of Flow |
| Open Source Code | Yes | The source code is available on Git Hub 1. 1https://github.com/Astuary/Flow |
| Open Datasets | Yes | (a) Stackoverflow [30] dataset is used for the next word prediction task... (b) Shakespeare [31] dataset is for the next character prediction task... (c) EMNIST [32] dataset is for 62-class image classification... (d-e) CIFAR10/100 [33] datasets are for 10and 100-class image classification tasks with Res Net18 [34] model. |
| Dataset Splits | Yes | Stackoverflow [30] dataset is used for the next word prediction task... The dataset contains a total of 342,477 train clients, whose combined sample count equals 135,818,730. Similarly, the dataset contains 38,758 validation and 204,088 test clients, whose combined sample counts equal 16,491,230 and 16,586,035, respectively. The Shakespeare dataset [31] consists of 715 distinct clients, each of which has its own training, validation, and test datasets. The combined training datasets of all clients contain a total of 12,854 instances, while the combined validation and test datasets contain 3,214 and 2,356 instances, respectively. |
| Hardware Specification | Yes | We use an NVidia 2080ti GPU to run all the experiments with 3 runs for each. |
| Software Dependencies | No | We use Flower [37] library to implement Flow and all its baselines. This statement does not include a version number for Flower. Other software like TensorFlow Federated Datasets are mentioned, but without version numbers. |
| Experiment Setup | Yes | The one layer LSTM we have used has 96 as embedding dimension, 670 as hidden state size, and 20 as the maximum sequence length [11]. The batch size used for each client on each baseline is 16. The vocabulary of this language task is limited to 10k most frequent tokens. The default learning rate used is 0.1. The number of clients per round is set to 10... For client-side training, the default epoch count is 3 for all the algorithms. For KNNPER, we used 5 nearby neighbors, and the mixture parameter is λ = 0.5. For APFL, mixture hyperparameter α is set to 0.25. DITTO has regularization hyperparameter λ = 0.1. There are 2 clusters by default for HYPCLUSTER. Flow and its variants were tested on the following choices of regularizing hyperparameters γ {1e-1, 1e-2, 1e-3, 1e-4}, where 1e-3 gave the best personalized accuracy. |