Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Annotator: A Generic Active Learning Baseline for LiDAR Semantic Segmentation
Authors: Binhui Xie, Shuang Li, Qingju Guo, Chi Liu, Xinjing Cheng
NeurIPS 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we conduct extensive experiments on several public benchmarks under three active learning scenarios: (i) AL setting where all available data points are from unlabeled target domain; (ii) ASFDA setting where we can only access a pre-trained model from the source domain; (iii) ADA setting where all data points from source domain can be utilized and a portion of unlabeled target data is selected to be annotated. We ο¬rst introduce the dataset used in this work and experimental setup and then present experimental results of baseline methods and extensive analyses of Annotator. |
| Researcher Affiliation | Collaboration | Binhui Xie Beijing Institute of Technology EMAIL Shuang Li Beijing Institute of Technology EMAIL Qingju Guo Beijing Institute of Technology EMAIL Chi Harold Liu Beijing Institute of Technology EMAIL Xinjing Cheng Tsinghua University & Inceptio Technology EMAIL |
| Pseudocode | No | The paper describes methods in text but does not contain structured pseudocode or algorithm blocks clearly labeled as such. |
| Open Source Code | No | Project page: https://binhuixie.github.io/annotator-web/ (As of the current date, the project page states 'Code coming soon...') |
| Open Datasets | Yes | We build all benchmarks upon Syn Li DAR [84], Semantic KITTI [3], Semantic POSS [42], and nu Scenes [4], constructing two simulation-to-real and two real-to-real adaptation scenarios. |
| Dataset Splits | Yes | Semantic KITTI (KITTI) [3] is a popular Li DAR segmentation dataset, including 2,9130 training scans and 6,019 validation scans with 19 categories. Semantic POSS (POSS) [42] consists of 2,988 real-world scans with point-level annotations over 14 semantic classes. As suggested in [42], we use the sequence 03 for validation and the remaining sequences for training. |
| Hardware Specification | Yes | all experiments share the same backbones and are within the same codebase, which are implemented using Py Torch [43] on a single NVIDIA Tesla A100 GPU. |
| Software Dependencies | No | The paper mentions 'implemented using Py Torch [43]' but does not specify a version number for PyTorch or any other software libraries or dependencies. Therefore, a reproducible description with specific version numbers is not provided. |
| Experiment Setup | Yes | We use the SGD optimizer and adopt a cosine learning rate decay schedule with initial learning rate of 0.01. And the batch size for both source and target data is 16. For additional details, please consult Appendix A.2. |