Enhancing Class-Imbalanced Learning with Pre-Trained Guidance through Class-Conditional Knowledge Distillation
Authors: Lan Li, Xin-Chun Li, Han-Jia Ye, De-Chuan Zhan
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on various imbalanced datasets demonstrate an average accuracy improvement of 7.4% using our method. |
| Researcher Affiliation | Academia | 1School of Artificial Intelligence, Nanjing University, China 2National Key Laboratory for Novel Software Technology, Nanjing University, China. |
| Pseudocode | Yes | Appendix B. Pseudo Code of ACCKD. Algorithm 1 Augmented Class-Conditional Knowledge Distillation (ACCKD) |
| Open Source Code | Yes | Our code is available at https://github.com/Lain810/CCKD. |
| Open Datasets | Yes | Dataset. We performed extensive experiments on three widely used imbalanced datasets: CIFAR-100-LT (Cui et al., 2019), Image Net-LT (Russakovsky et al., 2015), and i Naturalist 2018 (Horn et al., 2018). |
| Dataset Splits | Yes | It should be estimated using a validation set Dval that shares the same distribution as the test data... For CIFAR-100, following Cui et al. (2019), we created class imbalance by the imbalance ratio of Nmax/Nmin = 100. Image Net-LT consists of 115.8K images across 1000 classes, with a maximum of 1280 images and a minimum of 5 images per class. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or specific computing environments) used for running the experiments. |
| Software Dependencies | No | The paper mentions using Res Net-32 and Res Net-18 as backbone networks but does not specify any software libraries or their version numbers (e.g., PyTorch, TensorFlow, CUDA versions) required to reproduce the experiments. |
| Experiment Setup | Yes | We set the batch size to 128 and train for 200 epochs, using SGD as the optimizer. The initial learning rate of the optimizer is set to 0.1, with a momentum of 0.9 and a weight decay of 5 10 4. In terms of hyperparameters, we set the balancing parameters for the loss terms, α1 and α2, to 1 and 2, respectively. The temperature parameter for distillation, τ, is set to 2. |