MOKD: Cross-domain Finetuning for Few-shot Classification via Maximizing Optimized Kernel Dependence
Authors: Hongduan Tian, Feng Liu, Tongliang Liu, Bo Du, Yiu-Ming Cheung, Bo Han
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on Meta-Dataset demonstrate that MOKD can not only achieve better generalization performance on unseen domains in most cases but also learn better data representation clusters. |
| Researcher Affiliation | Academia | 1TMLR Group, Hong Kong Baptist University 2Department of Computer Science, Hong Kong Baptist University 3TMLR Group, University of Melbourne 4Sydney AI Centre, The University of Sydney 5National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, School of Computer Science, Wuhan Univeristy. |
| Pseudocode | Yes | A complete process of MOKD is summarized in Algorithm 1. |
| Open Source Code | Yes | The project repository of MOKD is available at: https://github.com/tmlr-group/MOKD. |
| Open Datasets | Yes | Extensive experiments on Meta-Dataset (Triantafillou et al., 2020) demonstrate that MOKD can not only achieve better generalization performance on unseen domains in most cases but also learn better data representation clusters. |
| Dataset Splits | Yes | Each dataset of the seen domain is divided into a training set, a validation set, and a test set roughly with the proportions of 75%, 15%, and 15%. |
| Hardware Specification | Yes | In this paper, all experiments are performed on an NVIDIA Ge Force RTX 3090 GPU. |
| Software Dependencies | No | The paper mentions using 'Adadelta (Zeiler, 2012)' and 'Res Net-18' (a model architecture) and implicitly 'PyTorch' due to the URL repository link, but no specific version numbers for software libraries or environments are provided for reproducibility. |
| Experiment Setup | Yes | For each adaptation episode, we re-initialize the linear transformation layer with an identity matrix and learn a set of task-specific parameters for the given task. The optimizer used in MOKD is Adadelta (Zeiler, 2012). The learning rate is 1.0 for Traffic Sign and MNIST and 0.25 for the remaining datasets. Besides, the weight decay is set to 0.25 for seen domains and 0.0 for unseen domains. Values of γ. In vary-way vary-shot task settings, we intuitively set γ to 1.0 for Omniglot, Aircraft, CU Birds, Quick Draw and MNIST while 3.0 for other datasets. In addition, in vary-way 5-shot and 5-way 1-shot task settings, we respectively set γ to 1.0 and 0 for all datasets since there are only a few data samples in each task. |