You Only Condense Once: Two Rules for Pruning Condensed Datasets
Authors: Yang He, Lingao Xiao, Joey Tianyi Zhou
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments validate our findings on networks including Conv Net, Res Net and Dense Net, and datasets including CIFAR10, CIFAR-100 and Image Net. For example, our YOCO surpassed various dataset condensation and dataset pruning methods on CIFAR-10 with ten Images Per Class (IPC), achieving 6.98-8.89% and 6.31-23.92% accuracy gains, respectively. The code is available at: https://github.com/he-y/you-only-condense-once. |
| Researcher Affiliation | Academia | CFAR, Agency for Science, Technology and Research, Singapore IHPC, Agency for Science, Technology and Research, Singapore {He_Yang, Joey_Zhou}@cfar.a-star.edu.sg |
| Pseudocode | Yes | Algorithm 1 Compute LBPE score for samples over epochs; Algorithm 2 Balanced Dataset Construction |
| Open Source Code | Yes | The code is available at: https://github.com/he-y/you-only-condense-once. |
| Open Datasets | Yes | The CIFAR-10 and CIFAR-100 datasets [20] are condensed via Conv Net-D3 [12], and Image Net-10 [5] via Res Net10-AP [13], both following IDC [19]. |
| Dataset Splits | No | The paper mentions training and testing, and uses 'validation' in the context of validating findings (e.g., 'Experiments validate our findings'), but does not specify validation dataset splits or how a validation set was explicitly used in the training process for hyperparameter tuning or early stopping. It only mentions 'The training batch size is 64, and the network is trained for 2000 × 100 epochs for CIFAR-10/CIFAR-100 and 500 × 100 epochs for Image Net-10.' and 'To reduce variance, we use the LBPE score from the top-K training epochs with the highest accuracy.' |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. It only mentions 'The computational work for this article was partially performed on resources of the National Supercomputing Centre (NSCC), Singapore (https://www.nscc.sg)' which is a general reference to a computing center without specific hardware details. |
| Software Dependencies | No | The paper mentions the use of 'SGD optimizer' and 'multi-step learning rate scheduler' but does not specify any software names with version numbers (e.g., Python, PyTorch, TensorFlow, or specific library versions). |
| Experiment Setup | Yes | For both networks, the learning rate is 0.01 with 0.9 momentum and 0.0005 weight decay. The SGD optimizer and a multi-step learning rate scheduler are used. The training batch size is 64, and the network is trained for 2000 × 100 epochs for CIFAR-10/CIFAR-100 and 500 × 100 epochs for Image Net-10. |