MIM4DD: Mutual Information Maximization for Dataset Distillation
Authors: Yuzhang Shang, Zhihang Yuan, Yan Yan
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we conduct comprehensive experiments to evaluate our proposed method MIM4DD on four different datasets for DD task. We first describe the implementation details of MIM4DD, and then compare our method with several So TA DD methods to demonstrate superiority of our proposed method. Finally, we validate the effectiveness of MI module (connected with Eq.2.16 and Eq.2.17) by a series of ablation studies. |
| Researcher Affiliation | Collaboration | Yuzhang Shang1, Zhihang Yuan2, Yan Yan1 1Department of Computer Science, Illinois Institute of Technology 2Huomo AI yshang4@hawk.iit.edu, zhihang.yuan@huomo.ai, yyan34@iit.edu |
| Pseudocode | No | The paper describes methods in prose and mathematical formulations but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Codes can be found anomalously in Supplement. |
| Open Datasets | Yes | We use MNIST [24], SVHN [35], and CIFAR10/100 datasets to conduct our experiments. MNIST [24] is a dataset for handwritten digits recognition... CIFAR10/100 [22] are two datasets consist of tiny colored natural images... |
| Dataset Splits | No | For MNIST: 'It contains 60,000 training images and 10,000 testing images'. For CIFAR10/100: 'In each dataset, 50,000 images are used for training and 10,000 images for testing.' While clear train/test splits are provided, the paper does not explicitly mention or specify a distinct validation dataset split. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments, such as CPU or GPU models, memory, or cloud instance types. |
| Software Dependencies | No | The paper mentions 'Convolutional Network (Conv Net)' and refers to models used in other papers but does not specify any software dependencies with version numbers (e.g., specific deep learning frameworks and their versions). |
| Experiment Setup | Yes | The synthetic images initial learning rate is 0.1, which is halved at the 1,800th and 2,800th iterations. The training is stopped after 5,000 iterations. To test the Conv Net s performance on the synthetic dataset, we train the network on synthetic sets for 300 epochs... The network s initial learning rate is 0.01. ... The default batch size is 256, and λ in Eq.2.17 is 0.8. |