Enhancing Meta Learning via Multi-Objective Soft Improvement Functions
Authors: Runsheng Yu, Weiyu Chen, Xinrun Wang, James Kwok
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical studies on various machine learning settings demonstrate that the proposed method is efficient, and achieves better performance than the baselines, particularly on improving the performance of the poorly-performing tasks and thus alleviating the compromising phenomenon. In this section, we perform experiments on few-shot regression (Section 5.1), few-shot classification (Section 5.2), and reinforcement learning (Section 5.3). |
| Researcher Affiliation | Academia | 1Department of Computer Science and Engineering, Hong Kong University of Science and Technology 2School of Computer Science and Engineering, Nanyang Technological University |
| Pseudocode | Yes | Algorithm 1: Soft Improvement Multi-Objective Meta Learning (SIMOL) Algorithms 2 and 3 show the pseudo-codes for SIMOL-based MAML and PN, respectively. |
| Open Source Code | No | Our implementations are based on the popular open-source meta-learning library Learn2Learn (Arnold et al., 2020). |
| Open Datasets | Yes | In this section, we perform 5-way-1-shot and 5-way-5-shot classification on the mini Image Net (Ravi & Larochelle, 2016) and tiered Image Net data (Ren et al., 2018). |
| Dataset Splits | Yes | Following (Finn et al., 2017), we split the mini Image Net dataset into a meta-training set with 64 classes, a meta-validation set with 16 classes, and a meta-testing set with 20 classes. Similarly, as in (Zhou et al., 2019), we split the tiered Image Net dataset into a meta-training set with 351 classes, a meta-validation set with 97 classes, and a meta-test set with 160 classes. |
| Hardware Specification | Yes | All experiments are run on a Ge Force RTX 2080 Ti GPU and Intel(R) Xeon(R) CPU E5-2680. |
| Software Dependencies | No | Our implementations are based on the popular open-source meta-learning library Learn2Learn (Arnold et al., 2020). |
| Experiment Setup | Yes | We use Adam (Kingma & Ba, 2014), with an initial learning rate of 0.01, to update the base learners for 5 steps in the inner-loop. ... The mini-batch size is 16. ... The initial learning rate for the outer loop is 0.001. The learning rates for the meta networks are 0.003 for MAML-based methods and 0.001 for PN-based methods, respectively. The learning rate of the reweighting network is 0.08 for SIMOL and 0.0008 for SIMOL+PN. The mini-batch size is 32. |