D-Flow: Differentiating through Flows for Controlled Generation
Authors: Heli Ben-Hamu, Omri Puny, Itai Gat, Brian Karrer, Uriel Singer, Yaron Lipman
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our framework on linear and non-linear controlled generation problems including: image and audio inverse problems and conditional molecule generation reaching state of the art performance across all. |
| Researcher Affiliation | Collaboration | 1Meta 2Weizmann Institute of Science. |
| Pseudocode | Yes | Algorithm 1 D-Flow. |
| Open Source Code | No | The paper mentions using the 'torchdiffeq package (Chen, 2018)' but does not explicitly state that the code for D-Flow itself is open-source or provide a link to it. |
| Open Datasets | Yes | We use the face-blurred Image Net128 dataset and report our results on the 10k split of the face-blurred Image Net dataset used by (Pokle et al., 2023). We evaluate on a subset of 1k samples from the validation set of the COCO dataset (Lin et al., 2015). The generative models used for this experiment are trained using the QM9 dataset (Ramakrishnan et al., 2014) |
| Dataset Splits | Yes | We use the face-blurred Image Net128 dataset and report our results on the 10k split of the face-blurred Image Net dataset used by (Pokle et al., 2023). We evaluate on a subset of 1k samples from the validation set of the COCO dataset (Lin et al., 2015). The train/validation/test partitions used are according to (Anderson et al., 2019) and consists of 100K/18K/13 samples per partition. |
| Hardware Specification | Yes | usually 5 15 minutes on Image Net-128 on an NVidia V100 GPU. Experiments ran on 32GB NVIDIA V100 GPU. On average, generating a single molecule took approximately 2.5 minutes using a single NVIDIA Quadro RTX8000 GPU. |
| Software Dependencies | No | The paper mentions using "torchdiffeq package (Chen, 2018)" and "Torch Metrics library (Detlefsen et al., 2022)" and "RDKit (Landrum, 2016)" but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | Yes | For all experiments in this section we used the LBFGS optimizer with 20 inner iterations for each optimization step with line search. Stopping criterion was set by a target PSNR value, varying for different tasks. The solver used was midpoint with 6 function evaluations. The losses, regularizations, initializations and stopping criterions of our algorithm for the linear inverse problems are listed in Table 6. We initialized the algorithm with a backward blend with α = 0.25. We employed the midpoint method for the ode solver, with a total of 100 function evaluations, i.e.step size of 1/50. The optimization technique utilized was LBFGS with line search, configured with 5 optimization steps and a limit of 5 inner iterations for each step. The learning rate was set to 1. |