Self-Consuming Generative Models Go MAD
Authors: Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, Richard Baraniuk
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct a thorough analytical and empirical analysis using state-of-the-art generative image models of three families of autophagous loops that differ in how fixed or fresh real training data is available through the generations of training and whether the samples from previous-generation models have been biased to trade off data quality versus diversity. |
| Researcher Affiliation | Academia | Department of ECE, Rice University; Department of Statistics, Stanford University; Department of CMOR, Rice University |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any statements about releasing open-source code or links to a code repository. |
| Open Datasets | Yes | We use an unconditional Style GAN2 model (Karras et al., 2020) and initially train it on n1 r = 70k samples from the FFHQ dataset (Karras et al., 2019b). ... We use a conditional DDPM (Ho et al., 2020) with T = 500 diffusion time steps and initially train it on n1 r = 60k real samples from the MNIST dataset. ... We consider the MNIST dataset as our reference distribution. |
| Dataset Splits | No | The training is carried out for 20 epochs with a batch size of 256 for each generation, ensuring convergence as determined by monitoring the model s likelihood over a validation set. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU models, CPU types, memory) used for running the experiments. |
| Software Dependencies | No | We use an unconditional Style GAN2 model (Karras et al., 2020)... We use a conditional DDPM (Ho et al., 2020)... For calculating FIDs, we use the features extracted by a Le Net (Lecun et al., 1998)... We implemented this example using the Invertible Networks.jl (Orozco et al., 2023) package for normalizing flows. |
| Experiment Setup | Yes | We use an unconditional Style GAN2 model (Karras et al., 2020) and initially train it on n1 r = 70k samples from the FFHQ dataset (Karras et al., 2019b). We downsized the FFHQ images to 128 × 128 (using Lanczos Py Torch anti-aliasing filtering as in Karras et al. (2020)) to reduce the computational cost. We set nt s = 70k for t ≥ 2. ... We use a conditional DDPM (Ho et al., 2020) with T = 500 diffusion time steps and initially train it on n1 r = 60k real samples from the MNIST dataset. We set nt s = 60k for t ≥ 2. ... The training is carried out for 20 epochs with a batch size of 256 for each generation, ensuring convergence as determined by monitoring the model s likelihood over a validation set. |