Inconsistency-Based Data-Centric Active Open-Set Annotation
Authors: Ruiyu Mao, Ouyang Xu, Yunhui Guo
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In contrast to recent learning-centric solutions, NEAT shows superior performance in active openset annotation, as our experiments confirm. Extensive experiments show that NEAT achieves much better results compared to standard active learning methods and the method specifically designed for active open set annotation. |
| Researcher Affiliation | Academia | Ruiyu Mao, Ouyang Xu, Yunhui Guo University of Texas at Dallas {ruiyu.mao, oxu, yunhui.guo}@utdallas.edu |
| Pseudocode | Yes | Algorithm 1: NEAT: Inconsistency-Based Data-Centric Active Open-Set annotation. |
| Open Source Code | No | Additional details on the further evaluation metrics, implementation, and architecture of our method can be found in the public document at https://arxiv.org/pdf/2401.04923.pdf. |
| Open Datasets | Yes | We consider CIFAR10 (Krizhevsky and Hinton 2009), CIFAR100 (Krizhevsky and Hinton 2009), and Tiny-Imagenet (Le and Yang 2015), to evaluate the performance of our proposed method. |
| Dataset Splits | No | The paper specifies how the initial labeled set is formed ( |
| Hardware Specification | Yes | The experiments were conducted on four A5000 NVIDIA GPUs. |
| Software Dependencies | No | The paper mentions using a Res Net-18 architecture, CLIP for feature extraction, and Stochastic Gradient Descent (SGD), but does not provide specific version numbers for any software dependencies like Python or PyTorch. |
| Experiment Setup | Yes | The classification model, specifically Res Net-18, was trained for 100 epochs using Stochastic Gradient Descent (SGD) with an initial learning rate of 0.01. The learning rate decayed by 0.5 every 20 epochs, and the training batch size is 128. There were 9 query rounds with a query batch size of 400. |