3D Indoor Instance Segmentation in an Open-World
Authors: Mohamed El Amine Boudjoghra, Salwa Al Khatib, Jean Lahoud, Hisham Cholakkal, Rao Anwer, Salman H. Khan, Fahad Shahbaz Khan
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments reveal the efficacy of the proposed contributions leading to promising open-world 3D instance segmentation performance. Code and splits are available at: https://github.com/aminebdj/3D-OWIS.5 Experiments5.1 Open-world evaluation protocol5.2 Implementation details5.3 Open-world results5.4 Incremental learning results5.5 Discussion and analysis |
| Researcher Affiliation | Academia | Mohamed El Amine Boudjoghra1, Salwa K. Al Khatib1, Jean Lahoud1, Hisham Cholakkal1, Rao Muhammad Anwer1,2, Salman Khan1,3, Fahad Shahbaz Khan1,4 1Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), 2Aalto University, 3Australian National University, 4Linköping University |
| Pseudocode | No | The paper does not contain any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | Code and splits are available at: https://github.com/aminebdj/3D-OWIS. |
| Open Datasets | Yes | We introduce carefully curated open-world splits, having known vs. unknown and then incremental learning over the span of 200 classes, for a rigorous evaluation of open-world 3D indoor segmentation. Our proposed splits leverage different realistic scenarios such as inherent distribution (frequency-based) of object classes, various class types encountered during the exploration of indoor areas (region-based), and the randomness aspect of object classes in the open-world.Table 1: The statistics of each split across the three tasks. The number of known classes per task is reported along with the count of instances (3D objects) in the training and validation set, we also show the number of non-empty scenes used during training and validation.We use our proposed splits of classes which mimic the challenges that are mostly faced in the open-world to ensure a strict performance evaluation for 3D instance segmentation models.We train the model on the entire Scan Net200 dataset for all tasks. |
| Dataset Splits | Yes | Table 1: The statistics of each split across the three tasks. The number of known classes per task is reported along with the count of instances (3D objects) in the training and validation set, we also show the number of non-empty scenes used during training and validation.Validation instances 6539 1000 428 4177 2261 1529 3776 2102 2089Validation scenes 312 242 165 312 264 236 307 273 300Evaluation: To conduct the evaluation during a task, we assign the 'unknown' label to the known classes from all the future tasks. |
| Hardware Specification | No | The paper mentions that 'The computational resources were provided by the National Academic Infrastructure for Supercomputing in Sweden (NAISS), partially funded by the Swedish Research Council through grant agreement No. 2022-06725, and by the Berzelius resource, provided by Knut and Alice Wallenberg Foundation at the National Supercomputer Center.' However, it does not specify any exact hardware models such as GPU or CPU types. |
| Software Dependencies | No | The paper mentions adapting 'Mask3D [29]' as a baseline and using a 'Minkowski backbone' for one of the comparison methods. However, it does not provide specific version numbers for any software, libraries, or frameworks (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | Yes | We adapt Mask3D [29] for the task of open-world instance segmentation. We add an extra prediction output for the unknown class. In training, we assign an ignore label to the classes of the future and previous tasks, while we keep the labels of the previous task and assign an unknown class label to the classes of the future task during evaluation. For contrastive clustering, we use the indices obtained after matching the predictions with the target using Hungarian matching to assign a label to the queries and store them in the Query Store Qstore. The store is then averaged per class and used to periodically update the prototypes every 10 iterations for the hinge loss computation. Finally, we use 40 exemplars per class on average for incremental learning. The classes from the current task are kept during class exemplar replay since we are using the same dataset for the three tasks. |