GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warping
Authors: Junyoung Seo, Kazumi Fukuda, Takashi Shibuya, Takuya Narihira, Naoki Murata, Shoukang Hu, Chieh-Hsin Lai, Seungryong Kim, Yuki Mitsufuji
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Qualitative and quantitative evaluations demonstrate that our model outperforms existing methods in both in-domain and out-of-domain scenarios. |
| Researcher Affiliation | Collaboration | Junyoung Seo1,3 Kazumi Fukuda1 Takashi Shibuya1 Takuya Narihira1 Naoki Murata1 Shoukang Hu1 Chieh-Hsin Lai1 Seungryong Kim3 Yuki Mitsufuji1,2 1Sony AI 2Sony Group Corporation 3KAIST AI |
| Pseudocode | No | The paper describes the method in text and with diagrams (e.g., Figure 3), but no formal pseudocode or algorithm blocks are provided. |
| Open Source Code | Yes | Project page is available at https://Gen Warp-NVS. github.io. |
| Open Datasets | Yes | We fine-tune the model on multi-view datasets including indoor scene and outdoor scene, i.e., Real Estate10K [52], Scan Net [8], ACID [25]. |
| Dataset Splits | No | The paper mentions using Real Estate10K [52], Scan Net [8], and ACID [25] for training, and tests on Real Estate10K and Scan Net. However, it does not explicitly provide specific details for a separate validation split, such as percentages or sample counts. |
| Hardware Specification | Yes | We initialize our two networks, semantic preserver and diffusion U-net, with Stable Diffusion v1.5 [35], and fine-tune the networks on 2 H100 80GB with a batch size of 48 for 2-3 days, at resolutions of 512 384 and 512 512. |
| Software Dependencies | Yes | We leverage the pretrained Stable Diffusion 1.5 model [35] for both diffusion U-net and semantic preserver network... Stable Diffusion v1.5 Model card: https://huggingface.co/runwayml/stable-diffusion-v1-5 |
| Experiment Setup | Yes | We initialize our two networks, semantic preserver and diffusion U-net, with Stable Diffusion v1.5 [35], and fine-tune the networks on 2 H100 80GB with a batch size of 48 for 2-3 days, at resolutions of 512 384 and 512 512. |