NeuMA: Neural Material Adaptor for Visual Grounding of Intrinsic Dynamics
Authors: Junyi Cao, Shanyan Guan, Yanhao Ge, Wei Li, Xiaokang Yang, Chao Ma
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Comprehensive experiments on various dynamics in terms of grounded particle accuracy, dynamic rendering quality, and generalization ability demonstrate that Neu MA can accurately capture intrinsic dynamics. |
| Researcher Affiliation | Collaboration | 1 Mo E Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University 2 vivo Mobile Communication Co., Ltd. {junyicao, xkyang, chaoma}@sjtu.edu.cn {guanshanyan, halege, liwei.yxgh}@vivo.com |
| Pseudocode | Yes | Algorithm 1: Particle Binding |
| Open Source Code | Yes | Project Page: https://xjay18.github.io/projects/neuma.html. |
| Open Datasets | Yes | For a comprehensive evaluation, we consider both synthetic and real-world data. Synthetic Data. Following PAC-Ne RF [45] and NCLaw [53], we use MPM [34] to simulate 6 kinds of dynamics... The generated 6 benchmarks are named Bouncy Ball , Jelly Duck , Rubber Pawn 1, Clay Cat , Honey Bottle , and Sand Fish . We report the simulation details in Appendix B. Real-world Data. We adopt the real-world data provided by Spring-Gaus [96] to assess the visual grounding performance in the real world. |
| Dataset Splits | Yes | To ensure an accurate object appearance and geometry, we additionally render 50 uniformly sampled viewpoints at t = 0, with the cameras evenly spaced on a sphere covering the object of interest, and use these data for static reconstruction. |
| Hardware Specification | Yes | All the experiments are conducted on a single NVIDIA A100 GPU. |
| Software Dependencies | No | The paper mentions software like Blender [9], RAdam optimizer [49], and MPM [34], but it does not provide specific version numbers for these software components or libraries. |
| Experiment Setup | Yes | We optimize the neural material adaptor Mθ using RAdam [49] optimizer with a cosine learning rate scheduler for 1, 000 iterations for each scene. We choose τ = 95% and ρ = 1, 000 for all experiments unless otherwise specified. Our MPM simulator operates in a [0, 1]3 cube with a fixed resolution of 323 and 703 for synthetic and real-world data unless otherwise specified. |