Score-Optimal Diffusion Schedules
Authors: Christopher Williams, Andrew Campbell, Arnaud Doucet, Saifuddin Syed
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate our proposed method on highly complex 1D distributions and show our method scales to high dimensional image data where it recovers previously known performant discretisation schedules discovered only through manual hyperparameter search. |
| Researcher Affiliation | Academia | Christopher Williams Department of Statistics University of Oxford Andrew Campbell Department of Statistics University of Oxford Arnaud Doucet Department of Statistics University of Oxford Saifuddin Syed Department of Statistics University of Oxford {williams,campbell,doucet,saifuddin.syed}@stats.ox.ac.uk |
| Pseudocode | Yes | Algorithm 1 Update Schedule |
| Open Source Code | Yes | We provide the code necessary to run our experiments. |
| Open Datasets | Yes | We train DDMs on CIFAR-10 and MNIST initialised at the cosine schedule... ; CIFAR-10 Krizhevsky et al. (2009) ; FFHQ Karras et al. (2018) ; AFHQv2 Choi et al. (2020) ; Image Net Deng et al. (2009). |
| Dataset Splits | No | The paper mentions training and testing on datasets like CIFAR-10 and MNIST but does not provide specific details on validation dataset splits (e.g., percentages or counts). |
| Hardware Specification | Yes | This takes on the order of 5 minutes to find the Corrector Optimised Schedule for CIFAR10 on a single RTX 2080Ti GPU. |
| Software Dependencies | No | The paper mentions using the codebase from Nichol and Dhariwal (2021) and Karras et al. (2022) but does not provide specific version numbers for software dependencies or libraries like PyTorch, TensorFlow, or CUDA versions. |
| Experiment Setup | Yes | For the MNIST experiments, we trained a model with an image size of 32, 32 channels, with a U-Net architecture, with 1 residual block per U-Net resolution, without learning sigma, and with a dropout rate of 0.3. The diffusion process was configured with 500 diffusion steps. Training was conducted with a learning rate of 1e-4. The batch size for MNIST was set to 128. |