EvoGrad: Efficient Gradient-Based Meta-Learning and Hyperparameter Optimization
Authors: Ondrej Bohdal, Yongxin Yang, Timothy Hospedales
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate Evo Grad on three substantial recent meta-learning applications, namely cross-domain few-shot learning with featurewise transformations, noisy label learning with Meta-Weight-Net and low-resource cross-lingual learning with meta representation transformation. The results show that Evo Grad significantly improves efficiency and enables scaling meta-learning to bigger architectures such as from Res Net10 to Res Net34. |
| Researcher Affiliation | Collaboration | Ondrej Bohdal1, Yongxin Yang1, Timothy Hospedales1,2 1 School of Informatics, The University of Edinburgh 2 Samsung AI Research Centre, Cambridge {ondrej.bohdal, yongxin.yang, t.hospedales}@ed.ac.uk |
| Pseudocode | No | The paper mentions an "Algorithm flow" but does not include a structured pseudocode or algorithm block. |
| Open Source Code | Yes | We provide source code for Evo Grad at https://github.com/ondrejbohdal/evograd. |
| Open Datasets | Yes | In this task we work with MNIST images [12] / (3.a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] Code and the instructions are a part of the supplemental material. |
| Dataset Splits | Yes | In this task we work with MNIST images [12], and assume that the validation and test sets are rotated by 30 compared to the conventionally oriented training images. / (3.b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] A short summary is provided in the main paper with further details in the supplemental material. |
| Hardware Specification | Yes | All our experiments are conducted on Titan X GPUs with 12GB of memory using K = 2 for Evo Grad. |
| Software Dependencies | No | The paper does not explicitly mention specific software dependencies with version numbers in the main text. |
| Experiment Setup | Yes | Further details and experimental settings are described in the supplementary material. / We use Evo Grad with 2 model copies, temperature τ = 0.05 and σ = 0.001 for ϵ σsign(N(0, I)). Our Le Net [13] base model is trained for 5 epochs. / Our second analysis studies the trajectories that parameters x, λ follow if they are both optimized online using SGD with learning rate of 0.1 for 5 steps, starting from five different positions (circles). |