Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views
Authors: Zixin Zou, Weihao Cheng, Yan-Pei Cao, Shi-Sheng Huang, Ying Shan, Song-Hai Zhang
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct experiments on CO3DV2 which is a multi-view dataset of real-world objects. Both quantitative and qualitative evaluations demonstrate that our approach outperforms previous state-of-the-art works on the metrics regarding NVS and geometry reconstruction. |
| Researcher Affiliation | Collaboration | Zixin Zou1, Weihao Cheng2, Yan-Pei Cao2, Shi-Sheng Huang3, Ying Shan2, Song-Hai Zhang1 1BNRist, Tsinghua University 2ARC Lab, Tencent PCG 3Beijing Normal University |
| Pseudocode | No | The paper describes its methods through text and diagrams (e.g., Figure 2 and 3) but does not include any explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | Ar Xiv version with supplementary materials is available at https://arxiv.org/abs/2308.14078 |
| Open Datasets | Yes | We conduct experiments on CO3DV2 which is a multi-view dataset of real-world objects. We follow the fewview-train and fewview-dev splits provided by CO3Dv2 dataset (Reizenstein et al. 2021) for training and evaluation purposes, respectively. |
| Dataset Splits | Yes | We follow the fewview-train and fewview-dev splits provided by CO3DV2 dataset (Reizenstein et al. 2021) for training and evaluation purposes, respectively. |
| Hardware Specification | Yes | Ne RF optimization runs for 10,000 steps, which takes about 45 minutes on a single 3090 GPU. |
| Software Dependencies | Yes | For the multiview-consistent model, we adopt the Stable Diffusion model v1.5 as our priors. For Ne RF reconstruction, we adapt the threestudio (Guo et al. 2023), which is a unified framework for 3D content creation from various inputs, to implement the Ne RF reconstruction for specific objects. |
| Experiment Setup | Yes | We set the weights of the losses with λp = 100, λc = 10, λr = 1000 and λm = 50. Ne RF optimization runs for 10,000 steps, which takes about 45 minutes on a single 3090 GPU. In our experiment with setting the CFG value as 7.5 |