Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Scaling Few-Shot Learning for the Open World
Authors: Zhipeng Lin, Wenjing Yang, Haotian Wang, Haoang Chi, Long Lan, Ji Wang
AAAI 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that the proposed SHA-Pipeline significantly outperforms not only the Proto Net baseline but also the state-of-the-art alternatives across different numbers of novel classes. |
| Researcher Affiliation | Collaboration | Zhipeng Lin1, Wenjing Yang1, Haotian Wang1, Haoang Chi2, 1, Long Lan1, Ji Wang1* 1 State Key Laboratory of High Performance Computing, National University of Defense Technology 2 Intelligent Game and Decision Lab, Academy of Military Science |
| Pseudocode | Yes | Algorithm 1: A Lightweight Parallel Framework, Algorithm 2: Algorithm of fine-tuning |
| Open Source Code | No | The paper does not provide any explicit statement or link for open-source code availability. |
| Open Datasets | Yes | We use the Image Net-1k((ILSVRC 2012-2017) as the base class set and construct a novel class dataset Image Net-21K-MNC based on Image Net-21K (winter 21 release), which consists of 19167 classes. |
| Dataset Splits | Yes | Among the remaining 16712 classes, 15000 classes are randomly selected for the meta-test and 1712 classes are used for the validation of meta-training. |
| Hardware Specification | Yes | All experiments are conducted on 8 NVIDIA A100 nodes with 8 GPUs each. |
| Software Dependencies | No | The paper mentions optimizers (SGD) and learning rate rules, but does not provide specific software versions for libraries like Python, PyTorch, CUDA, etc. |
| Experiment Setup | Yes | For meta-training, we use SGD optimizer without weight decay and a momentum of 0.9. The linear lr scaling rule of Goyal et al. (2017) are adopted: lr = base lr way / 5. For the learning rate schedule, we employ a cosine annealing learning rate schedule with a warm-up epoch of 5, from a base learning rate of 10-6 to 5 10-5. For Proto Net, P>M>F and Proto Net-Fix, the backbone network was meta-trained for 100 epochs, with each epoch consisting of 600 episodes. |