Synthetic data for model selection
Authors: Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, Matan Fintz, Gerard Medioni
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform extensive experiments on the CIFAR10 (Krizhevsky et al., 2009) dataset showing that indeed we are able to improve overall accuracy by selecting better models using synthetic data. |
| Researcher Affiliation | Industry | Alon Shoshan 1 Nadav Bhonker 1 Igor Kviatkovsky 1 Matan Fintz 1 G erard Medioni 1 1Amazon. Correspondence to: Alon Shoshan <alonshos@amazon.com>. |
| Pseudocode | No | The paper describes mathematical formulations and procedures like the linear ridge regression problem, but it does not present these in a structured pseudocode or algorithm block. |
| Open Source Code | No | The paper references third-party open-source projects like TIMM and pretrained-models.pytorch, and mentions that the generation code for Big GAN and DiT was publicly released by their respective authors, but it does not state that the code for the method described in *this* paper is open-source or available. |
| Open Datasets | Yes | In Sections 4.1 and 4.2, we perform experiments on CIFAR10 (Krizhevsky et al., 2009). ... In Section 4.3 we demonstrate rank preservation on Image Net (Deng et al., 2009). |
| Dataset Splits | Yes | In Sections 4.1 and 4.2, we perform experiments on CIFAR10 (Krizhevsky et al., 2009). In these sections, to evaluate the impact of the training set size, we use the following train-test splits: 10K-50K (Train10K), 30K-30K (Train30K), 50K-10K (Train50K). ... The standard protocol requires a validation set, to this end we split each of the datasets into training and validation subsets (train/val): Train10K was split into 7.5K/2.5K, Train30K was split into 22.5K/7.5K and Train50K was split into 40K/10K (see Appendix C for more details). |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., CPU, GPU models, or cloud computing instances) used for running the experiments. |
| Software Dependencies | No | The paper mentions software components such as StyleGAN2, WGAN-GP, DiT, BigGAN, TIMM, and PyTorch, but it does not specify their version numbers, which are necessary for reproducibility. |
| Experiment Setup | Yes | Appendix F (Standard Architectures Description) provides detailed hyperparameters for various deep model architectures, such as 'batch size 32, initial learning rate 0.05, depth 100, block type bottleneck, growth rate 12, compression rate 0.5' for DenseNet. Appendix G (Synthetic Data Generation Details (CIFAR10)) describes GAN training parameters, including how generators were selected based on FID scores. |