Few Shot Network Compression via Cross Distillation
Authors: Haoli Bai, Jiaxiang Wu, Irwin King, Michael Lyu3203-3210
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments n benchmark datasets demonstrate that cross distillation can significantly improve the student network s accuracy when only a few training instances are available. |
| Researcher Affiliation | Collaboration | 1The Chinese University of Hong Kong, 2Tencent AI Lab |
| Pseudocode | Yes | Algorithm 1 Cross distillation |
| Open Source Code | Yes | Our implementation in Py Torch is available at https://github.com/haolibai/ Cross-Distillation.git. |
| Open Datasets | Yes | evaluations are performed on CIFAR-10 and Image Net-ILSVRC12. |
| Dataset Splits | Yes | As we consider the setting of few shot image classification, we randomly select K-shot instances per class from the training set. All experiments are averaged over five runs with different random seeds, and results of means and standard deviations are reported. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU models, CPU types) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'Py Torch' but does not specify a version number or other software dependencies with versions. |
| Experiment Setup | Yes | For Ours, we choose μ = 0.6 for VGG networks and μ = 0.9 for Res Nets. For Ours-S, we set (α, β) = (0.9, 0.3) on VGG networks and (0.9, 0.5) on Res Nets. Sensitivity analysis on these hyper-parameters are presented later. |