Segment Anything in 3D with NeRFs
Authors: Jiazhong Cen, Zanwei Zhou, Jiemin Fang, chen yang, Wei Shen, Lingxi Xie, Dongsheng Jiang, XIAOPENG ZHANG, Qi Tian
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct various (e.g., object, part-level) segmentation tasks on the Replica [51] and NVOS [47] datasets. In this section, we quantitatively evaluate the segmentation ability of SA3D on various datasets. Then, we qualitatively demonstrate the versatility of SA3D, which can conduct instance segmentation, part segmentation, and text-prompted segmentation etc. |
| Researcher Affiliation | Collaboration | 1 Mo E Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University 2 Huawei Inc. 3 School of EIC, Huazhong University of Science and Technology |
| Pseudocode | No | The paper describes the method in prose and equations but does not include any explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/Jumpat/Segment Anythingin3D. |
| Open Datasets | Yes | For quantitative experiments, we use the Neural Volumetric Object Selection (NVOS) [47], SPIn Ne RF [39], and Replica [51] datasets. |
| Dataset Splits | No | By default, we utilize all available views in the training set I. The views are uniformly sampled from the sorted training set. |
| Hardware Specification | Yes | On an Nvidia RTX 3090 GPU, the 3D segmentation process with 5 views can be completed within 10 seconds. |
| Software Dependencies | No | We implement SA3D using Py Torch [42] with reference to the code provided by DVGOv2 [54]. |
| Experiment Setup | Yes | For our Ne RF model, we primarily employ the Tenso RF [3], utilizing the VM-48 representation to store the radiance latent vectors. The radiance fields are pre-trained for most datasets with 40,000 iterations. For the LLFF dataset [37] and the 360 dataset [1], the radiance fields are trained with 20,000 iterations. |