Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Differentially Private Meta-Learning
Authors: Jeffrey Li, Mikhail Khodak, Sebastian Caldas, Ameet Talwalkar
ICLR 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we apply our analysis to the problems of federated learning with personalization and few-shot classification, showing that allowing the relaxation to task-global privacy from the more commonly studied notion of local privacy leads to dramatically increased performance in recurrent neural language modeling and image classification. |
| Researcher Affiliation | Collaboration | Jeffrey Li, Mikhail Khodak, Sebastian Caldas Carnegie Mellon University EMAIL Ameet Talwalkar Carnegie Mellon University & Determined AI |
| Pseudocode | Yes | Algorithm 1: Online version of our (ε, δ)-meta-private parameter-transfer algorithm. |
| Open Source Code | No | The paper mentions using 'tools provided by TensorFlow Privacy' and links to its GitHub, but does not provide a link or statement about releasing the source code for their specific methodology. |
| Open Datasets | Yes | Datasets: We train a LSTM-RNN for next word prediction on two federated datasets: (1) The Shakespeare dataset as preprocessed by (Caldas et al., 2018), and (2) a dataset constructed from 3, 000 Wikipedia articles drawn from the Wiki-3029 dataset (Arora et al., 2019), where each article is used as a different task. For few-shot image classification, we use the Omniglot (Lake et al., 2011) and Mini-Image Net (Ravi and Larochelle, 2017) datasets |
| Dataset Splits | Yes | For Shakespeare, we set the number of tokens per task to 800 tokens, leaving 279 tasks for meta-training, 31 for meta-validation, and 35 for meta-testing. For Wikipedia, we set the number of tokens to 1, 600, which corresponds to having 2, 179 tasks for meta-training, 243 for meta-validation, and 606 for meta-testing. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU or CPU models, memory, or specific machine configurations used for running the experiments. |
| Software Dependencies | No | The paper mentions using 'TensorFlow Privacy' but does not specify its version number or any other software dependencies with version numbers. |
| Experiment Setup | Yes | Hyperparameters: For the language-modeling experiments, we tune the hyperparameters on the set of meta-validation tasks. For both datasets and all versions of the meta-learning algorithm, we tune hyperparameters in a two step process. We first tune all the parameters that are not related to refinement: the meta learning rate, the local (within-task) meta-training learning rate, the maximum gradient norm, and the decay constant. Then, we use the configuration with the best accuracy pre-refinement and then tune the refinement parameters: the refine learning rate, refine batch size, and refine epochs. |