PyNeRF: Pyramidal Neural Radiance Fields
Authors: Haithem Turki, Michael Zollhöfer, Christian Richardt, Deva Ramanan
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We first evaluate Py Ne RF s performance by measuring its reconstruction quality on bounded synthetic (Section 4.2) and unbounded real-world (Section 4.3) scenes. We demonstrate Py Ne RF s generalizability by evaluating it on additional Ne RF backbones (Section 4.4) and then explore the convergence benefits of using multiscale training data in city-scale reconstruction scenarios (Section 4.5). We ablate our design decisions in Section 4.6. |
| Researcher Affiliation | Collaboration | Haithem Turki Carnegie Mellon University hturki@cs.cmu.edu Michael Zollhöfer Meta Reality Labs Research zollhoefer@meta.com Christian Richardt Meta Reality Labs Research crichardt@meta.com Deva Ramanan Carnegie Mellon University deva@cs.cmu.edu |
| Pseudocode | Yes | Algorithm 1 Py Ne RF rendering function |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code for PyNeRF, nor does it include a link to a code repository. |
| Open Datasets | Yes | We evaluate Py Ne RF on the Multiscale Blender dataset proposed by Mip-Ne RF along with our own Blender scenes (which we name Blender-A )... We evaluate Py Ne RF s convergence properties on the the Argoverse 2 [35] Sensor dataset (to our knowledge, the largest city-scale dataset publicly available). |
| Dataset Splits | No | We implement Py Ne RF on top of the Nerfstudio library [28] and train on each scene with 8,192 rays per batch by default for 20,000 iterations on the Multiscale Blender and Mip-Ne RF 360 datasets, and 50,000 iterations on the Boat dataset and Blender-A... The resulting training set contains 400 billion rays across 150K video frames. |
| Hardware Specification | Yes | We train on each scene with 8,192 rays per batch by default for 20,000 iterations on the Multiscale Blender and Mip-Ne RF 360 datasets, and 50,000 iterations on the Boat dataset and Blender-A. We train a hierarchy of 8 Py Ne RF levels backed by a single multi-resolution hash table similar to that used by i NGP [20] in Section 4.2 and Section 4.3 before evaluating additional backbones in Section 4.4. We use 4 features per level with a hash table size of 220 by default, which we found to give the best quality-performance trade-off on the A100 GPUs we use in our experiments. |
| Software Dependencies | No | The paper mentions using the "Nerfstudio library [28]" but does not provide specific version numbers for Nerfstudio or other essential software dependencies like Python, PyTorch, or CUDA. |
| Experiment Setup | Yes | We implement Py Ne RF on top of the Nerfstudio library [28] and train on each scene with 8,192 rays per batch by default for 20,000 iterations on the Multiscale Blender and Mip-Ne RF 360 datasets, and 50,000 iterations on the Boat dataset and Blender-A. We train a hierarchy of 8 Py Ne RF levels backed by a single multi-resolution hash table similar to that used by i NGP [20] in Section 4.2 and Section 4.3 before evaluating additional backbones in Section 4.4. We use 4 features per level with a hash table size of 220 by default, which we found to give the best quality-performance trade-off on the A100 GPUs we use in our experiments. Each Py Ne RF uses a 64-channel density MLP with one hidden layer followed by a 128-channel color MLP with two hidden layers. We use similar model capacities in our baselines for fairness. We sample rays using an occupancy grid [20] on the Multiscale Blender dataset, and with a proposal network [3] on all others. We use gradient scaling [21] to improve training stability in scenes with that capture content at close distance (Blender-A and Boat). We parameterize unbounded scenes with Mip-Ne RF 360 s contraction method. |