Parameterized Knowledge Transfer for Personalized Federated Learning
Authors: Jie Zhang, Song Guo, Xiaosong Ma, Haozhao Wang, Wenchao Xu, Feijie Wu
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on various datasets (EMNIST, Fashion_MNIST, CIFAR-10) are conducted under different settings (heterogeneous models and data distributions). It is demonstrated that the proposed framework is the first federated learning paradigm that realizes personalized model training via parameterized group knowledge transfer while achieving significant performance gain comparing with state-of-the-art algorithms. |
| Researcher Affiliation | Academia | 1Department of Computing, The Hong Kong Polytechnic University 2Department of Computer Science and Technology, HUST {jieaa.zhang,harli.wu}@connect.polyu.hk, hz_wang@hust.edu.cn, {song.guo,xiaosma,wenchao.xu}@polyu.edu.hk |
| Pseudocode | Yes | Algorithm 1 KT-p FL Algorithm |
| Open Source Code | No | The paper does not provide an explicit statement or link for open-source code. |
| Open Datasets | Yes | We evaluate our proposed training framework on three different image classification tasks: EMNIST [35], Fashion_MNIST [36] and CIFAR-10 [37]. For each dataset, we apply two different Non-IID data settings: 1) each client only contains two classes of samples; 2) each client contains all classes of samples, while the number of samples for each class is different from that of a different client. All datasets are split randomly with 75% and 25% for training and testing, respectively. |
| Dataset Splits | No | The paper states 'All datasets are split randomly with 75% and 25% for training and testing, respectively' but does not specify a separate validation split or how it was handled. |
| Hardware Specification | Yes | We simulate a set of clients and a centralized server on one deep learning workstation (i.e., Intel(R) Core(TM) i9-9900KF CPU@3.6GHz with one NVIDIA Ge Force RTX 2080Ti GPU). |
| Software Dependencies | No | The experiments are implemented in Py Torch. However, no specific version number for PyTorch or any other software dependency is provided. |
| Experiment Setup | Yes | For all the methods and all the data settings, the batch size on private data and public data are 128 and 256, respectively, the number of local epochs is 20 and the distillation steps is 1 in each communication round of p FL training. Unless mentioned, otherwise the public data used for EMNIST and Fashion_MNIST is MNIST, and the public data used for CIFAR-10 is CIFAR-100. the size of public data used in each communication round is 3000, the learning rate is set to 0.01 for EMNIST and Fashion_MNIST, and 0.02 for CIFAR-10. |