Pre-Trained Model Reusability Evaluation for Small-Data Transfer Learning
Authors: Yao-Xiang Ding, Xi-Zhu Wu, Kun Zhou, Zhi-Hua Zhou
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that the MRE models learned by synergistic learning can generate significantly more reliable MRE scores than existing approaches for small-data transfer learning. |
| Researcher Affiliation | Academia | Yao-Xiang Ding State Key Lab for CAD&CG Zhejiang University yxding@zju.edu.cn Xi-Zhu Wu National Key Lab for Novel Software Technology Nanjing University wuxz@lamda.nju.edu.cn Kun Zhou State Key Lab for CAD&CG Zhejiang University kunzhou@acm.org Zhi-Hua Zhou National Key Lab for Novel Software Technology Nanjing University zhouzh@nju.edu.cn |
| Pseudocode | Yes | Algorithm 1 Synergistic Learning |
| Open Source Code | Yes | The code4 is implemented with Tensor Flow [Abadi et al., 2016] (Apache 2.0 License). More details of the experimental setups are discussed in Section C and more experimental results are included in Section D. 4The code is available on https://github.com/candytalking/Syn Learn. |
| Open Datasets | Yes | We adopt two ten-class datasets MNIST [Le Cun et al., 1998] and CIFAR-10 [Krizhevsky, 2009]... Next, we use the d Sprites dataset [Matthey et al., 2017]... Finally, we use CIFAR-100 [Krizhevsky, 2009] and Mini Image Net [Vinyals et al., 2016]. |
| Dataset Splits | No | The paper mentions 'Full training set is used for metric pre-training and 10% of the training set is used for metric learning' and 'its pre-defined training set is used for synergistic learning and its pre-defined testing set is used for testing.' for cross-dataset MRE. However, it does not explicitly define a 'validation' dataset split with specific percentages or sample counts for general model evaluation, relying on predefined training and testing sets or partial training sets for specific phases. |
| Hardware Specification | Yes | All experiments are conducted on servers with NVIDIA Tesla V100 GPUs. |
| Software Dependencies | No | The code4 is implemented with Tensor Flow [Abadi et al., 2016] (Apache 2.0 License). |
| Experiment Setup | Yes | We set training K = 10, but testing K = 1 for verifying the performance under the extremely challenging situation. All training and testing tasks are fixed to be fiveway classification. For each testing task, 50 instances are sampled from each class to test accuracy. |