How to Use Diffusion Priors under Sparse Views?
Authors: Qisen Wang, Yifan Zhao, Jiawei Ma, Jia Li
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on different public datasets show that our method achieves state-of-the-art reconstruction quality. |
| Researcher Affiliation | Academia | State Key Laboratory of Virtual Reality Technology and Systems, SCSE Beihang University {wangqisen, zhaoyf, majiawei, jiali}@buaa.edu.cn |
| Pseudocode | No | The paper provides mathematical equations and descriptions of methods, but no explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is released at https://github.com/i CVTEAM/IPSM. |
| Open Datasets | Yes | We evaluate our method on the LLFF [22] and DTU dataset [54]. |
| Dataset Splits | No | The paper specifies training and testing views but does not explicitly mention a separate validation split. It refers to '3 training views' and '15 testing scenes'. |
| Hardware Specification | Yes | All experimental results are obtained on a single RTX 3090. |
| Software Dependencies | Yes | All the experiments are conducted on a single RTX 3090 with CUDA 11.3. |
| Experiment Setup | Yes | The total training process involves 10K iterations for experiments on all datasets. The guidance of pseudo views starts from 2K iteration and ends at 9.5K iteration. The maximum degree of SH coefficients is set to 3, and we level up the SH degree every 500 iterations. For the inline prior, the mask threshold τ is set to 0.3 for IPSM regularization and 0.1 for the geometry consistency regularization. For the diffusion priors guidance, the weight λIPSM of IPSM regularization LIPSM is set to 2.0 for all datasets, and the parameter ηr for controlling LG1 IPSM and LG2 IPSM is set to 0.1 for all datasets. The parameter ηd for controlling the depth guidance of seen views and pseudo unseen views is set to 0.1 for all datasets. On the LLFF dataset [22], the weight λdepth of depth regularization Ldepth is set to 0.5 and the weight λgeo of the geometry consistency regularization Lgeo is set to 2.0. λssim is set to 0.2 and λ1 = 1 λssim following 3DGS [2]. On the DTU dataset [54], following DNGaussian [9], we reduce λ1 to 0.4 (i.e.increase λssim to 0.6), and at the same time reduce λdepth and λgeo, both of which are multiplied by 0.1. |