PhyRecon: Physically Plausible Neural Scene Reconstruction
Authors: Junfeng Ni, Yixin Chen, Bohan Jing, Nan Jiang, Bin Wang, Bo Dai, Puhao Li, Yixin Zhu, Song-Chun Zhu, Siyuan Huang
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that PHYRECON significantly improves the reconstruction quality. Our results also exhibit superior physical stability in physical simulators, with at least a 40% improvement across all datasets, paving the way for future physics-based applications. |
| Researcher Affiliation | Collaboration | 1 Tsinghua University 2 State Key Laboratory of General Artificial Intelligence, BIGAI 3 Peking University |
| Pseudocode | Yes | In Alg. 1, we illustrate the process of one training iteration of PHYRECON. |
| Open Source Code | Yes | https://phyrecon.github.io |
| Open Datasets | Yes | We conduct experiments on both the synthetic dataset Replica [62] and real datasets Scan Net [9] and Scan Net++ [76]. |
| Dataset Splits | No | The paper mentions using ScanNet, ScanNet++, and Replica datasets, but does not explicitly provide the train/validation/test splits used for these datasets. |
| Hardware Specification | Yes | All experiments are conducted on a single NVIDIA-A100 GPU. |
| Software Dependencies | No | The paper mentions PyTorch, Adam optimizer, and Diff Taichi but does not specify their version numbers. |
| Experiment Setup | Yes | We implement our model in Py Torch [53] and utilize the Adam optimizer [28] with an initial learning rate of 5e 4. We sample 1024 rays per iteration. When incorporating physics-guided pixel sampling, we allocate 768 rays for physics-guided pixel sampling and the remaining 256 rays for random sampling. Our model is trained for 450 epochs on Scan Net [9] and Scan Net++ [76] datasets, and 2000 epochs on Replica [62] dataset. As introduced in Sec. 3.4, training is divided into three stages. For the Scan Net [9] and Scan Net++ [76] datasets, the second and final stages begin at the 360th and 430th epochs, respectively, while for the Replica [62] dataset, these stages start at 1700th and 1980th epochs. All experiments are conducted on a single NVIDIA-A100 GPU. Following previous work [77, 32], we set 1, 0.1, 0.05, 0.04, 0.05, 0.1, 0.1 and 0.1 as loss weights for LRGB, LD, LN, LS, LEikonal, Lbs, Lop, Lrd, respectively. Additionally, we set ΞΎ = 100 for updating Gu-phy, initialize the loss weight for Lphy as 60, and increase it by 30 per epoch. |