Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Towards Enabling Meta-Learning from Target Models
Authors: Su Lu, Han-Jia Ye, Le Gan, De-Chuan Zhan
NeurIPS 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically verify the effectiveness of S/T protocol in a typical application of meta-learning, i.e., few-shot learning. |
| Researcher Affiliation | Academia | Su Lu Han-Jia Ye Le Gan De-Chuan Zhan State Key Laboratory for Novel Software Technology Nanjing University, Nanjing, 210023, China EMAIL, EMAIL |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/njulus/ST. |
| Open Datasets | Yes | In this part, we evaluate our S/T protocol on two benchmark datasets, i.e., mini Image Net [24] and tiered Image Net [17]. |
| Dataset Splits | Yes | 10000 tasks are used for both meta-training and meta-testing. 500 tasks are used for meta-validation. There are 64 classes for meta-training, 16 classes for meta-validation, and 20 classes for meta-testing. |
| Hardware Specification | Yes | We run the experiment on an Nvidia Ge Force RTX 2080ti GPU and Intel(R) Xeon(R) Silver 4110 CPU. |
| Software Dependencies | No | The paper mentions models like Res Net-12 and common frameworks are implied by the algorithms used (e.g., MAML, Proto Net), but it does not specify explicit software dependencies with version numbers (e.g., PyTorch 1.9, Python 3.8). |
| Experiment Setup | Yes | For each task, we generate 10 support instances by uniformly sampling x in range [ 5, 5]. For S/Q protocol, we additionally sample 30 query instances for each task. We set λ to 0.8, a relatively large value, in most of experiments. We pre-train a Res Net-12 with a linear layer on the meta-training split of mini Image Net. |