Adversarial Task Up-sampling for Meta-learning
Authors: Yichen WU, Long-Kai Huang, Ying Wei
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | On fewshot sine regression and image classification datasets, we empirically validate the marked improvement of ATU over state-of-the-art task augmentation strategies in the meta-testing performance and also the quality of up-sampled tasks. |
| Researcher Affiliation | Collaboration | Yichen Wu1,2 , Long-Kai Huang2 , Ying Wei1 1City University of Hong Kong, 2Tencent AI Lab |
| Pseudocode | Yes | We summarize the proposed ATU in Algorithm 1 in Appendix A. |
| Open Source Code | No | We exploit the open-source datasets and will release the code. |
| Open Datasets | Yes | We consider four datasets (base classes number): mini Imagenet-S (12), ISIC [18] (4), Dermnet-S (30), and Tabular Murris [5] (57) covering classification tasks on general natural images, medical images, and gene data. Note that the mini Imagenet-S and Dermnet-S are constructed by limiting the base classes of mini Imangenet [33] and Dermnet [1], respectively. |
| Dataset Splits | Yes | In the meta-training phase, each task contains K support and K target (K=10) examples. We adopt mean squared error (MSE) as the loss function. For the base model fθ, we adopt a small neural network, which consists of an input layer of size 1, 2 hidden layers of size 40 with Re LU and an output layer of size 1. We use one gradient update with a fixed step size α=0.01 in inner loop, and use Adam as the outer-loop optimizer following [9, 15]. Moreover, the meta-learner is trained on 240,000 tasks with meta batch-size being 4. |
| Hardware Specification | Yes | We use a workstation with Intel i7-2600 CPUs and NVIDIA GeForce RTX 2080 Ti GPUs. The training of the models takes roughly 2-3 days. |
| Software Dependencies | No | The paper mentions using 'Adam as the outer-loop optimizer' but does not specify software components with version numbers (e.g., Python, PyTorch/TensorFlow versions). |
| Experiment Setup | Yes | For the base model fθ, we adopt a small neural network, which consists of an input layer of size 1, 2 hidden layers of size 40 with Re LU and an output layer of size 1. We use one gradient update with a fixed step size α=0.01 in inner loop, and use Adam as the outer-loop optimizer following [9, 15]. Moreover, the meta-learner is trained on 240,000 tasks with meta batch-size being 4. |