Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
SelKD: Selective Knowledge Distillation via Optimal Transport Perspective
Authors: Liangliang Shi, Zhengyan Shi, Junchi Yan
ICLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on standard benchmarks demonstrate the superiority of our approach. The source code is available at: https://github.com/machoshi/Sel KD. [...] Our experiments are performed using Py Torch 1.4.0 and run on Intel Core i7-7820X CPU @ 3.60GHz with Nvidia Ge Force RTX 3080. We take single GPU for classification on CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009) and Tiny Image Net (Le & Yang, 2015), and evaluate on testing data by top-1 accuracy. [...] The results of Sel KD tasks on the CIFAR-100 and Tiny Image Net dataset are presented in Table 1 and Table 2, with results for CIFAR-10 is shown in the Appendix A. [...] Table 4: Ablation Study of Sel KD on CIFAR-100. |
| Researcher Affiliation | Academia | 1Sch. of Computer Science & Sch. of Artificial Intelligence, Shanghai Jiao Tong University 2Shanghai Artificial Intelligence Laboratory Equal Contributions Correspondence Author EMAIL |
| Pseudocode | Yes | Algorithm 1: Computing the KL divergence between student and teacher couplings under Open-set Sel KD |
| Open Source Code | Yes | Experimental results on standard benchmarks demonstrate the superiority of our approach. The source code is available at: https://github.com/machoshi/Sel KD. |
| Open Datasets | Yes | Our experiments are performed using Py Torch 1.4.0 and run on Intel Core i7-7820X CPU @ 3.60GHz with Nvidia Ge Force RTX 3080. We take single GPU for classification on CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009) and Tiny Image Net (Le & Yang, 2015), and evaluate on testing data by top-1 accuracy. |
| Dataset Splits | No | The paper describes how classes are divided into subtasks (e.g., "CIFAR-100 and Tiny Image Net, with 100 classes each, are divided into 5 subtasks, each containing 20 classes"), but it does not specify the train/test/validation dataset splits (e.g., 80/10/10 percentages or sample counts) for these datasets. It only mentions evaluating on "testing data" without detailing the splitting methodology for the main datasets. |
| Hardware Specification | Yes | Our experiments are performed using Py Torch 1.4.0 and run on Intel Core i7-7820X CPU @ 3.60GHz with Nvidia Ge Force RTX 3080. |
| Software Dependencies | Yes | Our experiments are performed using Py Torch 1.4.0 and run on Intel Core i7-7820X CPU @ 3.60GHz with Nvidia Ge Force RTX 3080. |
| Experiment Setup | Yes | As for the learning rate, we set 0.05 for all tasks with regard to CIFAR-10 and CIFAR-100 datasets. For Tiny Image Net, learning rate is 0.2. |