MGDD: A Meta Generator for Fast Dataset Distillation
Authors: Songhua Liu, Xinchao Wang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments demonstrate that the generator adapted with only a limited number of steps performs on par with those state-of-the-art DD methods and yields 22 acceleration. In this paper, we elaborate on the technical methods of the proposed MGDD pipeline. Our theoretical analysis reveals that the errors of synthetic datasets solved in the original space and then processed by any conditional generators are upper-bounded. |
| Researcher Affiliation | Academia | Songhua Liu Xinchao Wang National University of Singapore songhua.liu@u.nus.edu, xinchao@nus.edu.sg |
| Pseudocode | Yes | Algorithm 1 MGDD: Meta-Learning Framework for Training a Synthetic Sample Generator, Algorithm 2 Adaptation Algorithm of Synthetic Sample Generator for a Target Dataset |
| Open Source Code | No | No explicit statement or link confirming the release of MGDD's source code. |
| Open Datasets | Yes | we use Image Net1k [6], a large-scale image recognition dataset popular in the computer vision and machine learning communities; standard benchmarks, including CIFAR10 and CIFAR100 [23] datasets; Here, we conduct experiments on MNIST [25] and Fashion MNIST [52] datasets; PACS [29], two medical classification datasets Path MNIST and Blood MNIST [53], and one fine-grain image classification dataset CUB200 [48]. |
| Dataset Splits | No | There are 50,000 training images and 10,000 testing images in the 32 32 resolution for both datasets, and the numbers of classes are 10 and 100 respectively. (Only mentions training and testing images, no explicit validation split.) |
| Hardware Specification | Yes | We use a cloud server with a single A40 GPU for meta learning and a workstation with a single 3090 GPU for the subsequent adaptation. |
| Software Dependencies | No | The meta generator is trained by the Adam optimizer [21]... The implementation follows the open-source code of FRe Po [61]. (No specific versions for software libraries like PyTorch, Python, or CUDA are provided.) |
| Experiment Setup | Yes | For a clear view, we summarize the hyper-parameters and their values in both meta learning and adaptation stages as shown in Tab. 7. All experiments follow these default settings of hyper-parameters if not specified. Table 7 provides "Number of Meta Testing Steps T 200,000", "Learning Rate in Meta-Training α 1e-4", "Learning Rate of Generator α 1e-4", etc. |