Generative Sliced MMD Flows with Riesz Kernels
Authors: Johannes Hertrich, Christian Wald, Fabian Altekrüger, Paul Hagemann
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the efficiency of our model by image generation on MNIST, Fashion MNIST and CIFAR10. 5 NUMERICAL EXAMPLES In this section, we apply generative MMD flows for image generation on MNIST, Fashion MNIST (Xiao et al., 2017),CIFAR10 and Celeb A (Liu et al., 2015). The images from MNIST and Fashion MNIST are 28x28 gray-value images, while CIFAR10 consists of 32x32 RGB images resulting in the dimensions d = 784 and d = 3072, respectively. For Celeb A, we centercrop the images to 140x140 and then bicubicely resize them to 64x64. We run all experiments either on a single NVIDIA Ge Force RTX 3060 or a RTX 4090 GPU with 12GB or 24GB memory,respectively. To evaluate our results, we use the Fréchet inception distance (FID) (Heusel et al., 2017)1 between 10K generated samples and the test dataset. |
| Researcher Affiliation | Academia | Johannes Hertrich1, Christian Wald2, Fabian Altekrüger3, Paul Hagemann2 1 University College London, 2 Technische Universität Berlin, 3 Humboldt-Universität zu Berlin Correspondence to: j.hertrich@ucl.ac.uk |
| Pseudocode | Yes | Algorithm 1 Derivative of the interaction energy E from (5). Algorithm 2 Derivative of the potential energy V from (5). Algorithm 3 Training of generative MMD flows |
| Open Source Code | Yes | The code is available online at https://github.com/johertrich/sliced_MMD_flows. |
| Open Datasets | Yes | We demonstrate the efficiency of our generative sliced MMD flows for image generation on MNIST, Fashion MNIST and CIFAR10. ... We draw the first M = 20000 target samples from the MNIST training set and N = 20000 initial samples uniformly from [0, 1]d. |
| Dataset Splits | Yes | We stop the training of our generative sliced MMD flow when the FID between the generated samples and some validation samples does not decrease twice. Then we take the network with the best FID value to the validation set. The validation samples are the last 10000 training samples from the corresponding dataset which were not used for training the generative sliced MMD flow. |
| Hardware Specification | Yes | We run all experiments either on a single NVIDIA Ge Force RTX 3060 or a RTX 4090 GPU with 12GB or 24GB memory,respectively. |
| Software Dependencies | No | The paper mentions using UNets and Adam optimizer, but does not provide specific version numbers for any software libraries or dependencies, such as PyTorch, TensorFlow, or Python itself. It only references the papers where Adam and UNets were introduced/implemented. |
| Experiment Setup | Yes | The exact setup is described in Appendix H. ... We use UNets (Φ)L l=1 2 with 3409633 trainable parameters for MNIST and Fashion MNIST and 2064035 trainable paramters for CIFAR10. The networks are trained using Adam (Kingma & Ba, 2015) with a learning rate of 0.001. All flows are simulated with a step size τ = 1. ... MNIST. We draw the first M = 20000 target samples from the MNIST training set and N = 20000 initial samples uniformly from [0, 1]d. Then we simulate the momentum MMD flow using P = 1000 projections for 32 steps and train the network for 2000 optimizer steps with a batch size of 100. After each training of the network, we increase the number of flow steps by min(25+l, 2048) up to a maximal number of 30000 steps, where l is the iteration of the training procedure. We choose the momentum parameter m = 0.7 and stop the whole training after L = 55 networks. |