Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
Authors: Aniruddh Raghu, Maithra Raghu, Samy Bengio, Oriol Vinyals
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We investigate this question, via ablation studies and analysis of the latent representations, finding that feature reuse is the dominant factor. ANIL matches MAML s performance on benchmark few-shot image classification and RL and offers computational improvements over MAML. |
| Researcher Affiliation | Collaboration | Aniruddh Raghu MIT araghu@mit.edu Maithra Raghu Cornell University & Google Brain maithrar@gmail.com Samy Bengio Google Brain Oriol Vinyals Deep Mind |
| Pseudocode | No | The paper describes the MAML, ANIL, and NIL algorithms in prose and uses mathematical equations for formal definitions and examples, such as in Section 3.1 'OVERVIEW OF MAML' and Section C.1 'AN EXAMPLE OF THE ANIL UPDATE'. However, it does not provide any section explicitly labeled 'Pseudocode' or 'Algorithm' with structured, code-like steps. |
| Open Source Code | Yes | We used the Tensor Flow MAML implementation open-sourced by the original authors (Finn et al., 2017). We used the open source Py Torch implementation of MAML for RL 1, due to challenges encountered when running the open-sourced Tensor Flow... https://github.com/tristandeleu/pytorch-maml-rl |
| Open Datasets | Yes | We use the Mini Image Net dataset, a popular standard benchmark for few-shot learning... The Mini Imagenet dataset was proposed by Ravi and Larochelle (2016)... Omniglot: The Omniglot dataset consists of over 1600 different handwritten character classes from 23 alphabets. |
| Dataset Splits | Yes | The dataset is split on a character-level, so that certain characters are in the training set, and others in the validation set. (Omniglot) The Mini Imagenet dataset... consists of 64 training classes, 12 validation classes, and 24 test classes. |
| Hardware Specification | Yes | Experiments were run on a single NVIDIA Titan-Xp GPU. |
| Software Dependencies | No | The paper mentions using 'the Tensor Flow MAML implementation' and 'the open source Py Torch implementation of MAML for RL'. However, it does not specify version numbers for either TensorFlow, PyTorch, or any other ancillary software components. |
| Experiment Setup | Yes | Our models are trained using a batch size of 16, 5 inner loop updates, and an inner learning rate of 0.1. Our models are trained using a batch size of 4. 5 inner loop update steps, and an inner learning rate of 0.01 are used. |