Unraveling Model-Agnostic Meta-Learning via The Adaptation Learning Rate
Authors: Yingtian Zou, Fusheng Liu, Qianxiao Li
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | These results are corroborated with numerical experiments. and 5 EXPERIMENTS We verify our theorem through Neural Tangent Kernel (NTK) (Jacot et al., 2018) and deep learning on the Omniglot dataset (Lake et al., 2011). |
| Researcher Affiliation | Academia | Yingtian Zou, Fusheng Liu, Qianxiao Li National University of Singapore, Singapore {yingtian, fusheng}@u.nus.edu, qianxiao@nus.edu.sg |
| Pseudocode | No | No pseudocode or clearly labeled algorithm block was found. |
| Open Source Code | No | No explicit statement or link providing concrete access to source code for the described methodology was found. |
| Open Datasets | Yes | We verify our theorem through Neural Tangent Kernel (NTK) (Jacot et al., 2018) and deep learning on the Omniglot dataset (Lake et al., 2011). |
| Dataset Splits | Yes | MAML first adapts with an adaptation learning rate α on each task using its training set a subset of task data in the inner loop. Then, in the outer loop, MAML minimizes the evaluation loss for each adapted task-specific solution using a validation set. For simplicity, since data is i.i.d sampled from the same distribution, we first consider the setting where all data in each task is used as training set and validation set in our main results. We present later the extension of these results to the case with a different train-validation split. (Please refer to Appendix H.1) and If training feature Φ(u) Rk1 d is different from validation feature Φ(t) Rk2 d for every task, then... in Appendix H.1. |
| Hardware Specification | No | No specific hardware details (like exact GPU/CPU models, processor types, or memory amounts) were found that were used for running the experiments. It only mentions 'GPUs' in a general context in the acknowledgement. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., library or solver names with version numbers like Python 3.8, PyTorch 1.9, CPLEX 12.4) were found. |
| Experiment Setup | Yes | We set hyperparameters dimension d = 20, number of training/validation samples per task K = 50, number of tasks N = 5000. Each x is i.i.d sampled from a distribution U( 5, 5) while each a is i.i.d sampled from high dimension Gaussian distribution N(0, 3I). and The learning rate η in fast adaptation evaluation is 10 5. and All hyperparameters are set to be same as (Bernacchia, 2021) where... (further specific details in H.3) |