Articulate your NeRF: Unsupervised articulated object modeling via conditional view synthesis
Authors: Jianning Deng, Kartic Subr, Hakan Bilen
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our method on the synthetic 3D Part Net-Mobility dataset [43, 23, 3]... train the static Ne RF on 100 views from the first observation, train the part segmentation and articulation on 100 views from the second observation. We report the performance of our method for varying number of views from the second observation in Tab. 4. |
| Researcher Affiliation | Academia | Jianning Deng Kartic Subr Hakan Bilen University of Edinburgh |
| Pseudocode | No | The paper describes its procedures and steps within the main text and figures, but it does not include any explicitly labeled 'Pseudocode' or 'Algorithm' blocks or figures. |
| Open Source Code | Yes | https://github.com/VICO-Uo E/Articulate Your Nerf/ and At last, the code and the data used in this project will be released upon acceptance. |
| Open Datasets | Yes | We evaluate our method on the synthetic 3D Part Net-Mobility dataset [43, 23, 3]. While the dataset contains more than 2000 articulated objects from 46 different categories, we use a subset of the dataset with 6 shapes that was used in [17]. For a fair comparison, we downloaded the processed dataset from [17] which contains 2 sets of 100 views along with their foreground masks, each with a different articulation, and also groundtruth part segmentation labels, for each shape. |
| Dataset Splits | Yes | We train the static Ne RF on 100 views from the first observation, train the part segmentation and articulation on 100 views from the second observation. We report the performance of our method for varying number of views from the second observation in Tab. 4. Checkpoints with best PSNR during validation will be used for test. |
| Hardware Specification | Yes | Our experiments, requiring around 16 GB of VRAM, complete in approximately 30 minutes on a single RTX 4090 GPU for a single object. |
| Software Dependencies | No | The paper mentions using NeRF and related concepts but does not specify particular software dependencies (libraries, frameworks, or operating systems) with version numbers, which are required for a reproducible description. |
| Experiment Setup | Yes | As detailed in Sec. 4.2, we perform optimization on Mℓfor 4000 iterations and on sℓfor 2000 iterations for all evaluated objects. Checkpoints are saved following Step 3. We conduct this process through 5 cycles for objects with a single moving part and 6 cycles for objects with multiple moving parts. Checkpoints with best PSNR during validation will be used for test. During the first step of Mℓoptimization, we begin with a learning rate of 0.01, which linearly decays by 0.5 every 500 iterations. We accumulate gradients from 8 viewpoints to simulate a batch size of 8, initializing Mℓidentically in the first cycle and using the previously estimated Mℓfor subsequent cycles. In the second step of sℓoptimization using the Adam optimizer, the initial learning rate is set at 0.01 and linearly decays by a factor of 0.01 every 100 iterations. For multiple moving parts, initialization involves training the segmentation head s using pre-assigned labels on Xℓ, and querying predictions for x Xℓ. Cross-entropy loss optimized over 1000 iterations with a learning rate of 1e 3 shapes the learnable parameters in s. |