Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
MergeNet: Knowledge Migration Across Heterogeneous Models, Tasks, and Modalities
Authors: Kunxi Li, Tianyu Zhan, Kairui Fu, Shengyu Zhang, Kun Kuang, Jiwei Li, Zhou Zhao, Fan Wu, Fei Wu
AAAI 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on heterogeneous knowledge transfer demonstrate significant improvements in challenging settings, where representative approaches may falter or prove less applicable. [...] Our extensive experiments across various challenging scenarios validate the effectiveness of Merge Net. The results clearly demonstrate that Merge Net significantly improves model performance and surpasses the widelyused knowledge distillation techniques. |
| Researcher Affiliation | Academia | 1Zhejiang University 2Shanghai Jiao Tong University EMAIL,EMAIL |
| Pseudocode | No | The paper describes the Merge Net architecture and training process using mathematical formulations and descriptive text, but it does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not explicitly provide a link to a code repository, mention code in supplementary materials, or state that the code will be made publicly available. |
| Open Datasets | Yes | We conduct cross-structure knowledge transfer using the CIFAR-100 (Krizhevsky, Hinton et al. 2009) dataset. [...] We conduct experiments in cross-modal knowledge transfer on two distinct tasks: Visual Question Answering using the VQA v2.0 (Goyal et al. 2017) dataset and Image-Text Retrieval using the MSCOCO (Lin et al. 2014) dataset, with X-VLM (Zeng, Zhang, and Li 2021) as the experimental model. [...] We study the cross-task knowledge transfer effectiveness of our method on the following tasks: a classification task (IMDb sentiment classification (Maas et al. 2011)) and a question answering task (SQu AD v2.0 (Rajpurkar, Jia, and Liang 2018)). |
| Dataset Splits | Yes | This dataset comprises 100 categories, with training and validation sets containing 50k and 10k images, respectively. [...] Given the large size of the datasets and limited computational resources, which led to lengthy training times, we train the model using only 10% of the training set and assess the effectiveness of cross-modal knowledge transfer on the complete test set. |
| Hardware Specification | No | The paper mentions 'limited computational resources' but does not provide any specific hardware details such as GPU models, CPU types, or memory specifications used for the experiments. |
| Software Dependencies | No | The paper mentions using specific models like BERT and Distil BERT, but it does not specify any software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions, or other libraries). |
| Experiment Setup | Yes | We define the knowledge transfer cycle as Tcycle, during the time step t: As,t = M({As,t, Al,t}) if t mod Tcycle = 0, As,t otherwise. [...] Specifically, we schedule knowledge transfer to occur after every 2Tcycle batches in the question answering task and every Tcycle batches in the classification task. [...] Ws = Ws ηs Ws Ls({ Ws,t, Ws,u}), where ηs is the learning rate. |