A Mutual Information Perspective on Federated Contrastive Learning
Authors: Christos Louizos, Matthias Reisser, Denis Korzhenkov
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically evaluate our proposed extensions in various tasks to validate our claims and furthermore demonstrate that our proposed modifications generalize to other pretraining methods. Our experimental evaluation consist of unsupervised and semi-supervised experiments |
| Researcher Affiliation | Industry | Christos Louizos, Matthias Reisser, Denis Korzhenkov Qualcomm AI Research {clouizos,mreisser,dkorzhen}@qti.qualcomm.com |
| Pseudocode | Yes | Algorithm 1 The server side algorithm for our federated Sim CLR / Spectral CL / Sim Siam with optional user-verification and semi-supervision. Algorithm 2 The client side algorithm for our federated Sim CLR / Spectral CL / Sim Siam with optional user-verification and semi-supervision. |
| Open Source Code | No | The paper does not provide an explicit statement about open-sourcing the code for the described methodology, nor does it include a link to a code repository. |
| Open Datasets | Yes | For the label-skew setting, we use the Dirichlet splits for CIFAR 10, 100 discussed at Reddi et al. (2020) with α = 0.1 in both cases. Tiny Imagenet. It consists of 100k training examples and 10k test examples, each beloging to one of 200 classes. |
| Dataset Splits | No | The paper does not explicitly state specific training/test/validation dataset splits (e.g., 80/10/10 split or specific sample counts for a validation set) for reproducing the experiments. It mentions training and testing but lacks details on a distinct validation split. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, memory amounts, or specific cloud instances) used for running the experiments. It generally refers to 'resource constrained devices' but without specifying the hardware used for evaluation. |
| Software Dependencies | No | The paper mentions software components and optimizers like 'Adam Kingma & Ba (2014) optimizer' but does not provide specific version numbers for any software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | For local optimization we use standard stochastic gradient descent with a learning rate of 0.1 for both CIFAR 10 and CIFAR 100 for, unless mentioned otherwise, a single local epoch and a batch size of 128. After the local optimization on a specific round has been completed, each client communicates to the server the delta between the finetuned parameters and the model communicated from the server to the clients. The server averages these deltas, interprets them as gradients , and uses them in conjunction with the Adam Kingma & Ba (2014) optimizer in order to update the global model. |