Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision
Authors: Ayush Tewari, Tianwei Yin, George Cazenavette, Semon Rezchikov, Josh Tenenbaum, Fredo Durand, Bill Freeman, Vincent Sitzmann
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the efficacy of our approach on three challenging computer vision tasks. For instance, in inverse graphics, we demonstrate that our model in combination with a 3D-structured conditioning method enables us to directly sample from the distribution of 3D scenes consistent with a single 2D input image. |
| Researcher Affiliation | Academia | Ayush Tewari1 Tianwei Yin1 George Cazenavette1 Semon Rezchikov4 Joshua B. Tenenbaum1,2,3 Frédo Durand1 William T. Freeman1 Vincent Sitzmann1 1MIT CSAIL 2MIT BCS 3MIT CBMM 4Princeton IAS |
| Pseudocode | No | The paper illustrates steps of the method in figures and describes them in text, but does not provide formal pseudocode or algorithm blocks. |
| Open Source Code | No | Project page: diffusion-with-forward-models.github.io. This is a general project page, not an explicit statement of code release for the methodology, nor a direct link to a code repository. |
| Open Datasets | Yes | We evaluate on two challenging real-world datasets. We use Co3D hydrants [50] to evaluate our method on object-centric scenes. For scene-level 3D synthesis, we use the challenging Real Estate10k dataset [51], consisting of indoor and outdoor videos of scenes. |
| Dataset Splits | No | The paper mentions evaluating on datasets like Co3D and Real Estate10k and discusses metrics, but it does not specify explicit training, validation, or test dataset splits (e.g., percentages or sample counts). |
| Hardware Specification | No | The paper acknowledges that compute was provided by various groups ('Josh provided intriguing cognitive science perspectives... and provided a significant part of the compute. Vincent s Scene Representation Group provided a significant part of the compute'), but no specific hardware details such as GPU or CPU models are mentioned. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers, such as Python or library versions (e.g., PyTorch, TensorFlow). |
| Experiment Setup | No | The paper states 'We discuss these regularizers, as well as training details, in the supplement.' indicating that specific experimental setup details are not provided in the main text. |