Prompt-tuning Latent Diffusion Models for Inverse Problems
Authors: Hyungjin Chung, Jong Chul Ye, Peyman Milanfar, Mauricio Delbracio
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We test our method on the following degradations: 1) Super-resolution from 8 averagepooling, 2) Inpainting from 10-20% free-form masking as used in (Saharia et al., 2022a), 3) Gaussian deblurring from an image convolved with a 61 61 size Gaussian kernel with σ = 3.0, 4) Motion deblurring from an image convolved with a 61 61 motion kernel that is randomly sampled with intensity 0.54, following (Chung et al., 2023b). For all degradations, we include mild additive white Gaussian noise with σy = 0.01. |
| Researcher Affiliation | Collaboration | 1KAIST, Daejeon, Korea 2Google Research, Mountain View, US. |
| Pseudocode | Yes | We summarize our alternating sampling method in Algorithm 1 and Algorithm 2, based on DDIM sampling, with standard noise schedule notations adopted from (Ho et al., 2020). |
| Open Source Code | No | The paper mentions 'our jax implementation' but does not provide an explicit statement about open-sourcing the code or a link to a repository for the described methodology. |
| Open Datasets | Yes | We consider two different wellestablished datasets: 1) FFHQ 512 512 (Karras et al., 2019), and 2) Image Net 512 512 (Deng et al., 2009). |
| Dataset Splits | No | The paper mentions using '1k validation dataset' in table captions for evaluation, but the corresponding text indicates this is a selection of 1k images from the 10k ImageNet test images for testing. No explicit details are provided for how training, validation, or test splits were created from the primary datasets for the purpose of reproducing the experiment's training phase. |
| Hardware Specification | Yes | All experiments were done on NVIDIA A100 40GB GPUs. |
| Software Dependencies | No | The paper mentions using 'jax' for implementation but does not provide specific version numbers for JAX or any other software dependencies. |
| Experiment Setup | Yes | Guidance on hyperparameter selection P2L, with prompt embedding as an additional variable to optimize over, has more hyperparameters than standard DIS. While we report the best choices in Tab. 6, here we provide a solid choice that works well across most experiments. 1) For optimizing prompt embedding, 1 5 iterations (K) with a learning rate of 1e 4 yields stable performance... 2) One can reliably choose GD with static step size of 1.0 for LDPS update... 3) projection works when applied every 3-5 steps (γ), while the value of λ matters less and can be freely chosen between 0.1 1.0... |