Personalized Federated Learning through Local Memorization
Authors: Othmane Marfoq, Giovanni Neglia, Richard Vidal, Laetitia Kameni
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show on a suite of federated datasets that this approach achieves significantly higher accuracy and fairness than state-of-the-art methods. |
| Researcher Affiliation | Collaboration | 1Inria, Universit e Cˆote d Azur, Sophia Antipolis, France 2Accenture Labs, Sophia Antipolis, France. |
| Pseudocode | Yes | Algorithm 1 k NN-Per (Typical usage) Learn global model using available clients with Fed Avg. for each client m [M] (in parallel) do Build datastore using Sm. At inference on x X, return hm,λm (x) given by (7) end for |
| Open Source Code | Yes | Code is available at https: //github.com/omarfoq/knn-per. |
| Open Datasets | Yes | We evaluate k NN-Per on four federated datasets spanning a wide range of machine learning tasks: language modeling (Shakespeare (Caldas et al., 2018; Mc Mahan et al., 2017)), image classification (CIFAR-10 and CIFAR100 (Krizhevsky, 2009)), handwritten character recognition (FEMNIST (Caldas et al., 2018)). |
| Dataset Splits | Yes | For FEMNIST and Shakespeare, we randomly split each local dataset into training (60%), validation (20%), and test (20%) sets. For CIFAR-10 and CIFAR-100, we maintained the original training/test data split and used 20% of the training dataset as validation dataset. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions using the 'FAISS library' and 'Proto NN' but does not specify their version numbers or any other software dependencies with versions. |
| Experiment Setup | Yes | In all experiments with CIFAR-10 and CIFAR-100, training spanned 200 rounds with full clients participation at each round for all methods. The learning rate was reduced by a factor 10 at round 100 and then again at round 150. For Shakespeare, 10% of clients were sampled uniformly at random without replacement at each round, and we trained for 300 rounds with a constant learning rate. For FEMNIST, 5% of the clients participated at each round for a total 1000 rounds, with the learning rate dropping by a factor 10 at round 500 and 750. |