Efficient Variance Reduction for Meta-learning
Authors: Hansi Yang, James Kwok
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on benchmark few-shot classification data sets demonstrate its effectiveness over state-of-the-art meta-learning algorithms with and without variance reduction. |
| Researcher Affiliation | Academia | 1Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong. |
| Pseudocode | Yes | Algorithm 1 STORM (Cutkosky & Orabona, 2019). |
| Open Source Code | No | The paper does not provide any concrete access information for source code. |
| Open Datasets | Yes | Experiments are performed on Mini Imagenet (Vinyals et al., 2016; Ravi & Larochelle, 2017) and Meta-Dataset (Triantafillou et al., 2020). |
| Dataset Splits | No | Table 1. Statistics for the data sets used. number of classes training validation testing #samples per class. The paper indicates general categories in Table 1 but doesn't specify how they partitioned the data for their experiments (e.g., specific percentages or sample counts for train/val/test splits). |
| Hardware Specification | No | The paper does not provide specific hardware details used for running its experiments. |
| Software Dependencies | No | we use vanilla SGD as the optimizer for the outer loop, and Adam for the inner loop. No specific version numbers for software dependencies are provided. |
| Experiment Setup | Yes | For all data sets, the hyper-parameter settings follow Reptile (Nichol et al., 2018): we use vanilla SGD as the optimizer for the outer loop, and Adam for the inner loop. The learning rate for SGD is 1, and no momentum is used. The learning rate for Adam is 0.001, the first-order momentum weight is 0, and the second-order momentum weight is 0.99. The number of gradient descent steps K in the inner loop is 5. |