Adaptive Compositional Continual Meta-Learning
Authors: Bin Wu, Jinyuan Fang, Xiangxiang Zeng, Shangsong Liang, Qiang Zhang
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show ACML outperforms strong baselines, showing the effectiveness of our compositional meta-knowledge, and confirming that ACML can adaptively learn meta-knowledge. |
| Researcher Affiliation | Academia | 1Zhejiang University, Hangzhou, China 2University of Glasgow, Glasgow, UK 3Hunan University, Changsha, China 4Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE 5Sun Yat-sen University, Guangzhou, China. |
| Pseudocode | Yes | Algorithm 1 The meta-training process of ACML. |
| Open Source Code | Yes | Our code is publicly available https://github.com/BinWu-Cs/AC-CML. |
| Open Datasets | Yes | Following exiting works (Yap et al., 2021; Zhang et al., 2021), we conduct the experiments on four datasets: VGGFlowers(Nilsback & Zisserman, 2008), mini Imagenet(Ravi & Larochelle, 2017), CIFAR-FS(Bertinetto et al., 2018), and Omniglot(Lake et al., 2011). |
| Dataset Splits | Yes | mini Imagenet: mini Imagenet(Ravi & Larochelle, 2017) is designed for few-shot learning, which consists of 100 different classes. Similarly, we also split the dataset into three datasets (i.e., 64 classes for meta-training, 16 classes for validation and 20 classes for meta-test) following the existing works. |
| Hardware Specification | Yes | We ran our algorithm on NVIDIA Tesla V100 32GB GPU. |
| Software Dependencies | No | The paper mentions using Adam and SGD optimizers, but does not provide specific version numbers for software dependencies like Python, PyTorch, or other libraries. |
| Experiment Setup | Yes | For our model, we use the Adam optimizer as the outer optimizer and the SGD optimizer as the inner optimizer. For the Monte Carlo sampling used in our algorithm, we set the number of sampling as 5. For the initial number of components in the compositional meta-knowledge, we set it as 4. All the important hyperparameters can be seen in Tab. 5. Table 5 includes: The number of outer update step 2000, The outer learning rate 0.001, The number of outer update step 3, The inner learning rate 0.05. |