Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data
Authors: Gongfan Fang, Yifan Bao, Jie Song, Xinchao Wang, Donglin Xie, Chengchao Shen, Mingli Song
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate Mosaic KD over classification and semantic segmentation tasks across various benchmarks, and demonstrate that it yields results much superior to the state-of-the-art counterparts on OOD data. |
| Researcher Affiliation | Collaboration | 1Zhejiang University, 2National University of Singapore, 3Central South University 4Alibaba-Zhejiang University Joint Institute of Frontier Technologies |
| Pseudocode | Yes | Algorithm 1 Mosaic KD for out-of-domain knowledge distillation |
| Open Source Code | Yes | Our code is available at https://github.com/zju-vipa/Mosaic KD. |
| Open Datasets | Yes | Four datasets are considered in our experiments as in-domain training set, including CIFAR-100 [26], CUB-200 [53], Stanford Dogs [24] and NYUv2 [34]. For OODKD, we substitute original training data with OOD data, including CIFAR-10 [26], Places365 [63], Image Net [9] and SVHN [36]. |
| Dataset Splits | No | The paper states, "More details about datasets, training protocol, and metrics can be found in supplementary materials." However, explicit training/test/validation dataset splits (e.g., percentages, sample counts, or explicit reference to predefined splits in the main text) are not provided within the paper's main content. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware used for running its experiments, such as GPU models, CPU types, or cloud computing specifications. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies or libraries used in the experiments. |
| Experiment Setup | No | The paper mentions "training protocol" details are in supplementary materials but does not provide specific hyperparameters (e.g., learning rate, batch size, number of epochs, optimizer settings) or other explicit configuration steps for the experimental setup in the main text. |