Impartial Adversarial Distillation: Addressing Biased Data-Free Knowledge Distillation via Adaptive Constrained Optimization
Authors: Dongping Liao, Xitong Gao, Chengzhong Xu
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive experiments, we show that our method mitigates the biased learning of majority classes in DFKD and improves the overall performance compared with baselines. Code will be available at https://github.com/ldpbuaa/ipad. We evaluate IPAD against the prevalent DFKD methods on multiple image classification datasets that are frequently used in recent DFKD literature (Do et al. 2022; Fang et al. 2021b,a), including CIFAR-100 (Krizhevsky, Hinton et al. 2009), Tiny-Image Net (Le and Yang 2015), Food101 (Bossard, Guillaumin, and Van Gool 2014), Places365 (Zhou et al. 2017) and Image Net (Deng et al. 2009). |
| Researcher Affiliation | Academia | Donping Liao1*, Xitong Gao2*, Chengzhong Xu1 1 State Key Lab of Io TSC, Department of Computer and Information Science, University of Macau, Macau SAR, China 2 Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China yb97428@um.edu.mo, xt.gao@siat.ac.cn, czxu@um.edu.mo |
| Pseudocode | No | The paper describes a "primal-dual algorithm" and "primal-dual updating scheme" but does not provide structured pseudocode or an algorithm block in the main text. Details are deferred to Appendix E. |
| Open Source Code | No | Code will be available at https://github.com/ldpbuaa/ipad. The statement "Code will be available" indicates a future release, which according to the guidelines, results in a 'No'. |
| Open Datasets | Yes | We evaluate IPAD against the prevalent DFKD methods on multiple image classification datasets that are frequently used in recent DFKD literature (Do et al. 2022; Fang et al. 2021b,a), including CIFAR-100 (Krizhevsky, Hinton et al. 2009), Tiny-Image Net (Le and Yang 2015), Food101 (Bossard, Guillaumin, and Van Gool 2014), Places365 (Zhou et al. 2017) and Image Net (Deng et al. 2009). |
| Dataset Splits | No | The paper describes how imbalanced datasets were created and refers to training strategies, but it does not explicitly provide details about specific training, validation, and testing dataset splits (e.g., percentages or sample counts for each partition). |
| Hardware Specification | No | The paper does not provide any specific details regarding the hardware used for conducting the experiments (e.g., CPU, GPU models, memory, or cloud computing instance types). |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers (e.g., programming languages, libraries, or frameworks like Python, PyTorch, or TensorFlow versions). |
| Experiment Setup | No | The paper mentions general aspects of the experimental setup such as model architectures (e.g., WRN40-2, WRN16-2, ResNet34, VGG16), the use of Cross-Entropy loss, and the creation of imbalanced datasets with varying imbalance ratios (r). However, it lacks specific quantitative hyperparameters (e.g., learning rate, batch size, number of epochs, optimizer settings) in the main text, often deferring such details to appendices (e.g., "Detailed experiement setups are outlined in Appendix D"). |