Federated-EM with heterogeneity mitigation and variance reduction
Authors: Aymeric Dieuleveut, Gersende Fort, Eric Moulines, Geneviève Robin
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical results are presented to support our theoretical findings, as well as an application to federated missing values imputation for biodiversity monitoring. |
| Researcher Affiliation | Academia | Aymeric Dieuleveut Centre de Math ematiques Appliqu ees Ecole Polytechnique, France Institut Polytechnique de Paris aymeric.dieuleveut@polytechnique.edu Gersende Fort Institut de Math ematiques de Toulouse Universit e de Toulouse; CNRS gersende.fort@math.univ-toulouse.fr Eric Moulines Centre de Math ematiques Appliqu ees Ecole Polytechnique, France CS Dpt, HSE University, Russian Federation eric.moulines@polytechnique.edu Genevi eve Robin Laboratoire de Math ematiques et Mod elisation d Evry Universit e d Evry Val d Essonne; CNRS Evry-Courcouronnes, France genevieve.robin@cnrs.fr |
| Pseudocode | Yes | Algorithm 1: Fed EM with partial participation and Algorithm 2: VR-Fed EM |
| Open Source Code | No | The paper does not provide any statement or link indicating that the source code for the methodology is openly available. |
| Open Datasets | Yes | MNIST Data set. We perform a similar experiment on the MNIST dataset to illustrate the behaviour of Fed EM and VR-Fed EM on a GMM inference problem with real data. We apply Fed Miss EM to the analysis of part of the e Bird data base [34, 1] of field observations reported in France by I = 2,465 observers, across J = 9,721 sites and at L = 525 monthly time points. |
| Dataset Splits | No | The paper describes data distribution across servers and minibatch usage but does not explicitly provide training/validation/test dataset splits or mention a 'validation' set. |
| Hardware Specification | No | The paper mentions 'edge devices' and 'central server' but does not provide specific hardware details (e.g., CPU/GPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies (e.g., library or solver names with version numbers) needed to replicate the experiments. |
| Experiment Setup | Yes | For Fed EM, we consider the finite-sum setting when si = m^-1 Pm j=1 sij with m = 10^2; the oracle Sk+1,i is obtained by a sum over a minibatch of b = 20 examples. For VR-Fed EM, we set b = 5 and kin = 20. We run the two algorithms for 500 epochs (one epoch corresponds to N conditional expectation evaluations sij). We choose b = 20 and the step size is constant and set to γ = 10^-3. We run Fed Miss EM for 150 epochs; with γ = 10^-4, = 10^-3, b = 10^2, a rank r = 2 and λ = 0; for the distribution of the variables Xi j , we use a Gaussian distribution with unknown expectation j and variance 1. |