Monocular Dynamic View Synthesis: A Reality Check
Authors: Hang Gao, Ruilong Li, Shubham Tulsiani, Bryan Russell, Angjoo Kanazawa
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive evaluation on existing datasets [5, 7] as well as a new dataset that includes more challenging motion and diverse scenes. In this section, we conduct a series of empirical studies to disentangle the recent progress in dynamic view synthesis (DVS) given a monocular video from effective multi-view in the training data. |
| Researcher Affiliation | Collaboration | Hang Gao1, Ruilong Li1 Shubham Tulsiani2 Bryan Russell3 Angjoo Kanazawa1 1UC Berkeley 2Carnegie Mellon University 3Adobe Research |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code and data can be found at http://hangg7.com/dycheck. |
| Open Datasets | Yes | We investigate the datasets used for evaluation in D-Ne RF [3], Hyper Ne RF [7], Nerfies [5], and NSFF [4]. Table 1 shows their statistics. We evaluate the amount of effective multi-view cues via the proposed EMFs, shown in Figure 3. We find that existing datasets have large EMF values on both metrics. ... We also propose a new dataset called the i Phone dataset shown in Figure 7. |
| Dataset Splits | No | We train all approaches under these two settings with the same held-out validation frames and same set of co-visibility masks computed from common training frames. (This indicates validation is used, but not the specific split sizes or percentages). |
| Hardware Specification | No | The paper does not explicitly mention specific hardware details such as GPU/CPU models, memory, or cloud instance types used for experiments. |
| Software Dependencies | No | We consolidate Nerfies [5] and Hyper Ne RF [7] in one codebase using JAX [46]. (The reference [46] points to 'JAX: composable transformations of Python+Num Py programs, 2018.' which gives a year but not a specific version number of the software itself.) |
| Experiment Setup | Yes | We consolidate Nerfies [5] and Hyper Ne RF [7] in one codebase using JAX [46]. Compared to the original official code releases, our implementation aligns all training and evaluation details between models and allows correspondence readout. Concretely, we consider the following: (+B) random background compositing [32]; (+D) a depth loss on the ray matching distance [1, 4]; and (+S) a sparsity regularization for scene surface [49]. |