On Kinetic Optimal Probability Paths for Generative Models
Authors: Neta Shaul, Ricky T. Q. Chen, Maximilian Nickel, Matthew Le, Yaron Lipman
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We further support this theory with empirical experiments on Image Net. In this section we: (i) approximate the data separation function λ numerically for different real-world datasets, and approximate the corresponding Kinetic Optimal (KO) paths for these datasets. (ii) We validate our theory of the convergence of λ 1 in high dimensions for real-world data. (iii) We empirically test our KO paths compared to paths defined by the conditional probabilities of Cond-OT (Lipman et al., 2022), IS (Albergo & Vanden-Eijnden, 2022), and DDPM (Ho et al., 2020). |
| Researcher Affiliation | Collaboration | Neta Shaul 1 Ricky T. Q. Chen 2 Maximilian Nickel 2 Matt Le 2 Yaron Lipman 2 1Weizmann Institute of Science 2Meta AI (FAIR). |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide an explicit statement about open-sourcing the code for its methodology or a link to a code repository. |
| Open Datasets | Yes | In terms of datasets, we have been experimenting with a 2D dataset (checkerboard), and the image datasets CIFAR10, Image Net-32, and Imagenet-64. For Image Net we use the official face-blurred Image Net and downsample to 32 × 32 using an open source preprocessing script (Chrabaszcz et al., 2017). |
| Dataset Splits | No | The paper mentions using "3 different epochs at the final stage of training" for FID and NFE, and lists training parameters in Table 2, but it does not specify explicit train/validation/test dataset splits (e.g., percentages or counts) or refer to standard predefined splits. |
| Hardware Specification | Yes | In Table 4 we present the runtime of computing λ for k = 1 and n = 50, 000 on 1 GPU (Quadro RTX 8000) for a number of image datasets of different image sizes. |
| Software Dependencies | No | The paper mentions using ADAM optimizer, Cosine Annealing scheduler, and a U-Net architecture, but it does not provide specific version numbers for any software dependencies or libraries. |
| Experiment Setup | Yes | For the CNF training in Section 6.3 we model vt(x) as follows: (i) For the 2D data it is an MLP with 5 layers consisting of 512 neurons in each layer. (ii) For the image datasets (CIFAR, Image Net-32), we used U-Net architecture as in (Dhariwal & Nichol, 2021), where the particular architecture hyper-parameters are detailed in Table 2. In this table we also provide training hyper-parameters. |