Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Data
Authors: Pei-Yau Weng, Minh Hoang, Lam Nguyen, My T. Thai, Lily Weng, Nghia Hoang
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our reported results on a variety of computer vision datasets confirm that the proposed method is most effective to combat extreme data heterogeneity in federated learning. |
| Researcher Affiliation | Collaboration | Pei-Yau Weng Washington State University pei-yau.weng@wsu.edu Minh Hoang Princeton University minhhoang@princeton.edu Lam M. Nguyen IBM Research lamnguyen.mltd@ibm.com My T. Thai University of Florida mythai@cise.ufl.edu Tsui-Wei Weng University of California San Diego lweng@ucsd.edu Trong Nghia Hoang Washington State University trongnghia.hoang@wsu.edu |
| Pseudocode | Yes | Algorithm 1 Probabilistic Federated Prompt Tuning (PFPT) input: pre-trained model F, no. τ of iterations, no. m of sampled clients per iteration output: optimized set of prompts Φ |
| Open Source Code | Yes | Code Release. Our experimental code is released and maintained at https://github.com/Pei Yau Weng/PFPT. |
| Open Datasets | Yes | Our experiments are conducted on a variety of computer vision datasets, including CIFAR10 and CIFAR-100 [51], Tiny Image Net [52] and a synthetic, diverse dataset created by pooling together the MNIST-M [53], Fashion-MNIST [54], CINIC-10 [55] and MMAFEDB (available on Kaggle) datasets, which is referred to as the 4-dataset. ... All the used datasets are publicly available. |
| Dataset Splits | No | The paper mentions 'train partition' and 'test partition' but does not explicitly state a separate 'validation' split for model tuning or early stopping. |
| Hardware Specification | Yes | All experiments are performed on a V100 GPU with 32GB GPU RAM. |
| Software Dependencies | No | The paper mentions optimizers like Adam and SGD but does not provide specific version numbers for software libraries, frameworks (e.g., PyTorch, TensorFlow), or programming languages used. |
| Experiment Setup | Yes | The hyperparameter settings are all presented in Table 9. Table 9: Hyperparameter setting for all baselines and our PFPT Method Setting Batch size Communication round Eps. in local training Optimizer & learning rate Total clients Sampled clients |