Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Probabilistic Federated Prompt-Tuning with Non-IID and Imbalanced Data
Authors: Pei-Yau Weng, Minh Hoang, Lam Nguyen, My T. Thai, Lily Weng, Nghia Hoang
NeurIPS 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our reported results on a variety of computer vision datasets confirm that the proposed method is most effective to combat extreme data heterogeneity in federated learning. |
| Researcher Affiliation | Collaboration | Pei-Yau Weng Washington State University EMAIL Minh Hoang Princeton University EMAIL Lam M. Nguyen IBM Research EMAIL My T. Thai University of Florida EMAIL Tsui-Wei Weng University of California San Diego EMAIL Trong Nghia Hoang Washington State University EMAIL |
| Pseudocode | Yes | Algorithm 1 Probabilistic Federated Prompt Tuning (PFPT) input: pre-trained model F, no. τ of iterations, no. m of sampled clients per iteration output: optimized set of prompts Φ |
| Open Source Code | Yes | Code Release. Our experimental code is released and maintained at https://github.com/Pei Yau Weng/PFPT. |
| Open Datasets | Yes | Our experiments are conducted on a variety of computer vision datasets, including CIFAR10 and CIFAR-100 [51], Tiny Image Net [52] and a synthetic, diverse dataset created by pooling together the MNIST-M [53], Fashion-MNIST [54], CINIC-10 [55] and MMAFEDB (available on Kaggle) datasets, which is referred to as the 4-dataset. ... All the used datasets are publicly available. |
| Dataset Splits | No | The paper mentions 'train partition' and 'test partition' but does not explicitly state a separate 'validation' split for model tuning or early stopping. |
| Hardware Specification | Yes | All experiments are performed on a V100 GPU with 32GB GPU RAM. |
| Software Dependencies | No | The paper mentions optimizers like Adam and SGD but does not provide specific version numbers for software libraries, frameworks (e.g., PyTorch, TensorFlow), or programming languages used. |
| Experiment Setup | Yes | The hyperparameter settings are all presented in Table 9. Table 9: Hyperparameter setting for all baselines and our PFPT Method Setting Batch size Communication round Eps. in local training Optimizer & learning rate Total clients Sampled clients |