Unsupervised Meta-learning via Few-shot Pseudo-supervised Contrastive Learning
Authors: Huiwon Jang, Hankook Lee, Jinwoo Shin
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our extensive experiments demonstrate that Ps Co outperforms existing unsupervised meta-learning methods under various in-domain and cross-domain few-shot classification benchmarks. |
| Researcher Affiliation | Collaboration | AKorea Advanced Institute of Science and Technology (KAIST) BLG AI Research {huiwoen0516, jinwoos}@kaist.ac.kr hankook.lee@lgresearch.ai |
| Pseudocode | Yes | Algorithm 1 Pseudo-supervised Contrast (Ps Co): Py Torch-like Pseudocode |
| Open Source Code | Yes | The code is available at https://github.com/alinlab/Ps Co. |
| Open Datasets | Yes | Omniglot (Lake et al., 2011) is a 28 × 28 gray-scale dataset of 1623 characters with 20 samples each. Mini Image Net (Ravi & Larochelle, 2017) is an 84 × 84 resized subset of ILSVRC-2012 (Deng et al., 2009) with 600 samples each. |
| Dataset Splits | Yes | We split the dataset into 120, 100, and 323 classes for meta-training, meta-validation, and meta-test respectively. In addition, the 0, 90, 180, and 270 degrees rotated views for each class become the different categories. Thus, we have a total of 6492, 400, and 1292 classes for meta-training, meta-validation, and meta-test respectively. (Omniglot) |
| Hardware Specification | Yes | In our experiments, we mainly use NVIDIA GTX3090 GPUs. |
| Software Dependencies | No | The paper mentions 'Pytorch' but does not provide a specific version number. It also references 'torchvision (Paszke et al., 2019)' without a version. |
| Experiment Setup | Yes | We train our models via stochastic gradient descent (SGD) with a batch size of N = 256 for 400 epochs. Following Chen et al. (2020b); Chen & He (2021), we use an initial learning rate of 0.03 with the cosine learning schedule, τMo Co = 0.2, and a weight decay of 5 × 10−4. We use a queue size of M = 16384 since Omniglot (Lake et al., 2011) and mini Image Net (Ravi & Larochelle, 2017) has roughly 100k meta-training samples. |