Rethinking Centered Kernel Alignment in Knowledge Distillation
Authors: Zikai Zhou, Yunhang Shen, Shitong Shao, Linrui Gong, Shaohui Lin
IJCAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The extensive experiments on the CIFAR-100, Image Net-1k, and MS-COCO demonstrate that our method achieves state-of-the-art performance on almost all teacher-student pairs for image classification and object detection, validating the effectiveness of our approaches. |
| Researcher Affiliation | Collaboration | School of Computer Science and Technology, East China Normal University, Shanghai, China Key Laboratory of Advanced Theory and Application in Statistics and Data Science, MOE, China Youtu Lab, Tencent, Shanghai, China Hunan University, China |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available in https://github.com/Klayand/PCKA. |
| Open Datasets | Yes | To validate the effectiveness of our approaches, we conduct extensive experiments on image classification (CIFAR100 [Krizhevsky and Hinton, 2009] and Image Net-1k [Russakovsky et al., 2015]), and object detection (MS-COCO [Lin et al., 2014]) tasks. |
| Dataset Splits | Yes | Results on the CIFAR-100 test set. Results on the Image Net validation set. Results on the COCO validation set |
| Hardware Specification | No | The paper mentions training with "2 or 4 per GPU" but does not specify any particular GPU model, CPU, or other hardware details. |
| Software Dependencies | No | The paper mentions using "Torchvision" but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | We apply a batch size of 128 and an initial learning rate of 0.1 for the SGD optimizer on CIFAR-100. And we follow the settings in [Huang et al., 2022] for the Res Net34-Res Net18 pair and the Res Net50-Mobile Net pair on Image Net-1k. The settings of other classification and detection tasks are in Appendix B. |