Generative Modelling with Inverse Heat Dissipation
Authors: Severi Rissanen, Markus Heinonen, Arno Solin
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We showcase generative sequences, quantitative evaluation, and analyse the noise hyperparameters σ and δ. We then study emergent properties, starting with the overall colour and other features becoming disentangled. Next, contrary to standard diffusion models, interpolations in the full latent u1:K are smooth. We also show that the forward heat process induces structure to the function learned by the neural net. Finally, we show that the model can generalise just from the first 20 MNIST digits. |
| Researcher Affiliation | Academia | Severi Rissanen, Markus Heinonen & Arno Solin Department of Computer Science Aalto University severi.rissanen@aalto.fi |
| Pseudocode | Yes | We summarize the training process in Alg. 1, and the sampling process in Alg. 2, both of which are straightforward to implement. |
| Open Source Code | Yes | Code for the methods in this paper is available at: https://github.com/Aalto ML/generative-inverse-heat-dissipation. |
| Open Datasets | Yes | We use K = 100 iteration steps on MNIST (Le Cun et al., 1998) 200 steps on CIFAR-10 (Krizhevsky, 2009), AFHQ (Choi et al., 2020), and FFHQ (Karras et al., 2019), and 400 on LSUN-CHURCHES (Yu et al., 2015). |
| Dataset Splits | No | The paper mentions calculating FID scores and validation loss but does not explicitly provide the specific percentages or counts for training, validation, and test splits needed for reproduction. It mentions using 'the training set to calculate the reference statistics' and 'the entire data sets' for FID, but not explicit train/val splits for model training. |
| Hardware Specification | Yes | We use NVIDIA A100 GPUs for the experiments, with two GPUs for 256 256 resolution models and one GPU for all others. |
| Software Dependencies | No | The paper mentions using 'Pytorch automatic mixed precision functionality', 'Adam optimizer', and 'clean-fid (Parmar et al., 2022)' but does not provide specific version numbers for any of these software components. |
| Experiment Setup | Yes | Table 1: Neural network hyperparameters. (e.g., Base channels, Learning rate, Batch size, EMA, # Blocks). Table 2: Hyperparameters related to the generative process. (e.g., K, σB,max, σ, δ). |