Resilient Constrained Learning
Authors: Ignacio Hounie, Alejandro Ribeiro, Luiz F. O. Chamon
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We showcase the advantages of this resilient learning method in image classification tasks involving multiple potential invariances and in heterogeneous federated learning. Our final contribution is the experimental evaluation of the resilient learning algorithm: (C3) We evaluate resilient formulations of federated learning and invariant learning (Section 5). Our experiments show that (C1) (C2) effectively relaxes constraints according to their difficulty, leading to solutions that are less sensitive to the requirement specifications. |
| Researcher Affiliation | Academia | Ignacio Hounie University of Pennsylvania Alejandro Ribeiro University of Pennsylvania Luiz F. O. Chamon University of Stuttgart |
| Pseudocode | Yes | Algorithm 1 Resilient Constrained Learning (η, ηu, ηλ > 0; θ(0) Θ; λ(0), u(0) Rm +), Algorithm 2 Resilient Federated Learning |
| Open Source Code | No | The paper does not state that source code for the described methodology is released or provide a link to a code repository. |
| Open Datasets | Yes | All of the plots in section 5 correspond to CIFAR10 [59]. Results for Fashion MNIST [49] can be found on F.4.4. We showcase our approach on datasets with artificial invariances, following the setup of [23]. Explicitly, we generate the synthetic datasets, by applying either rotations, translations, or scalings, to each sample in the MNIST [48] and Fashion MNIST [49] datasets. |
| Dataset Splits | No | The paper mentions creating a 'test set for each client with the same class balance as the train set' and refers to 'Train Test Split' in Figure 2, but it does not specify any validation set split sizes or methodologies. |
| Hardware Specification | No | The paper does not specify any particular hardware used for running the experiments (e.g., CPU, GPU models, or cloud computing instances). |
| Software Dependencies | No | The paper mentions using 'Fed AVG [56]' and 'Fed PD [57]' as algorithms, and 'Adam [54]' for optimization, but it does not list specific software dependencies with version numbers (e.g., 'Python 3.8', 'PyTorch 1.9'). |
| Experiment Setup | Yes | In all experiments we use 100 clients, and all clients participate from each communication round. We run 500 communication rounds in total. We set ϵ = 0.1, dual learning rate ηλ = 0.1, local learning rate ηθ = 5 × 10−2 and use no data augmentation in all experiments. In the resilient learning formulation, unless stated otherwise, we use a quadratic penalty on the perturbation h(u) = α ∥u∥^2_2 and a perturbation learning rate ηu = 0.1. |