Intermediate Layer Optimization for Inverse Problems using Deep Generative Models
Authors: Giannis Daras, Joseph Dean, Ajil Jalal, Alex Dimakis
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically show that our approach outperforms state-of-the-art methods introduced in Style GAN-2 and PULSE for a wide range of inverse problems including inpainting, denoising, super-resolution and compressed sensing. |
| Researcher Affiliation | Academia | 1The University of Texas at Austin. |
| Pseudocode | Yes | Algorithm 1 ILO for one layer of the generator |
| Open Source Code | No | The paper does not provide an explicit statement about releasing their code or a direct link to a code repository for the ILO method. |
| Open Datasets | Yes | To quantify the performance of the different methods we randomly select a few images from Celeba HQ (Liu et al., 2018; Lee et al., 2020) and reconstruct at different levels of sparsity. To demonstrate this, we run the following experiment; we remove entirely the loss functions that relate the generated images with a reference image (i.e. MSE and LPIPS) and we add a new classification loss term using an external classifier trained on a different domain. Essentially, we search for latent codes that lie in an l1 ball around the range of intermediate layers and maximize the probability that the generated image belongs to a certain category. We consider a classifier trained on Image Net (Deng et al., 2009). |
| Dataset Splits | No | The paper describes using Celeba-HQ and ImageNet datasets and discusses reconstruction errors, but it does not specify explicit training, validation, or test splits (e.g., percentages or exact counts) for these datasets. |
| Hardware Specification | Yes | To obtain the plots, we use a single V100 GPU. |
| Software Dependencies | No | The paper mentions software like Style GAN-2, PULSE, DCGAN, Big GAN, BM3D, and various loss functions (LPIPS, MSE), but it does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | To ensure that we stay in an l1 ball around the manifold at each layer, we use Projected Gradient Descent (PGD) (Nesterov, 2003). To implement the projection to an l1 ball around the current best solution (see line 4 of Algorithm (1)), we use the method of Duchi et al. (2008). Guided by our theory, we increase the maximum allowed deviation as we move to higher dimensional latent spaces. The radii of the balls are tuned separately as hyperparameters, for a full description see the Appendix. Style GAN-2 typically requires 300 1000 optimization steps (on the first layer) for a good reconstruction (Karras et al., 2019; 2020). However, we observe that running 50 steps in each one of the first four layers outperforms CSGM. We run 300 optimization steps per layer. The leftmost point corresponds to CSGM, i.e. we optimize over only the first layer. |