PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification
Authors: Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, Chuang Gan
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate the effectiveness of our proposed framework on geometry and physical parameter estimation over a vast range of materials, including elastic bodies, plasticine, sand, Newtonian and non-Newtonian fluids, and demonstrate significant performance gain on most tasks1.We conduct various experiments to study the efficacy of PAC-Ne RF on system identification tasks and find that: PAC-Ne RF can recover high-quality object geometries solely from videos. PAC-Ne RF performs significantly better on system identification tasks compared to fully learned approaches. PAC-Ne RF alleviates the assumptions that other techniques require (i.e., known object geometry), while outperforming them. Purely pixel-based loss functions provide rich gradients that enable physical parameter estimation. |
| Researcher Affiliation | Collaboration | Xuan Li1, , Yi-Ling Qiao2, Peter Yichen Chen3,4, Krishna Murthy Jatavallabhula3, Ming Lin2, Chenfanfu Jiang1, Chuang Gan5, 6 1UC Los Angeles, 2University of Maryland, 3MIT CSAIL, 4Columbia University, 5UMass Amherst, 6MIT-IBM Watson AI Lab |
| Pseudocode | No | The paper describes the methodology in prose and mathematical equations (e.g., Equation 1, 2, 3, 4, 5, 6), but does not contain a distinct section or figure explicitly labeled as 'Pseudocode' or 'Algorithm'. |
| Open Source Code | No | Demos are available on the project webpage: https://sites.google.com/view/PAC-Ne RF |
| Open Datasets | No | Dataset: To evaluate different system identification methods, we simulate and render a wide range of objects using a photo-realstic simulation engine with varying environment lighting conditions and ground textures. Our dataset includes deformable objects, plastics, granular media, Newtonian and non-Newtonian fluids. Figure 3 demonstrates a few example scenarios. |
| Dataset Splits | No | We first train a static voxel Ne RF using data from the first frame (following Sun et al. (2022)) with the Adam optimizer.For each material type discussed above, we generate 10 problem instances by varying object orientations, initial velocities, and physical parameters. The comparisons are conducted on this dataset. |
| Hardware Specification | Yes | The entire training takes 1.5 hours on a single Nvidia 3090 GPU. |
| Software Dependencies | No | Our differentiable MPM is using Diff Taichi (Hu et al., 2020). |
| Experiment Setup | Yes | The architecture of voxel discretization of Ne RF follows Sun et al. (2022), which stores density value and color feature within 1603 voxels, only contraining an extra 2-layer MLP with a hidden dimension 128 for view-dependent color fields. The dimension of color feature on each voxel is 12.We first train a static voxel Ne RF using data from the first frame (following Sun et al. (2022)) with the Adam optimizer. The initial velocity estimator uses L-BFGS, which we experimentally find to be better than Adam for this sub-task. For all other physical parameters of interest, we use the Adam optimizer. |