Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Meta-Query-Net: Resolving Purity-Informativeness Dilemma in Open-set Active Learning
Authors: Dongmin Park, Yooju Shin, Jihwan Bang, Youngjun Lee, Hwanjun Song, Jae-Gil Lee
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on multiple open-set active learning scenarios demonstrate that the proposed MQ-Net achieves 20.14% improvement in terms of accuracy, compared with the state-of-the-art methods. |
| Researcher Affiliation | Collaboration | Dongmin Park1, Yooju Shin1, Jihwan Bang2,3, Youngjun Lee1, Hwanjun Song2 , Jae-Gil Lee1 1 KAIST, 2 NAVER AI Lab, 3 NAVER CLOVA |
| Pseudocode | Yes | The pseudocode of MQ-Net can be found in Appendix B. |
| Open Source Code | Yes | The code is available at https://github.com/kaist-dmlab/MQNet. |
| Open Datasets | Yes | We perform the active learning task on three benchmark datasets; CIFAR10 [41], CIFAR100 [41], and Image Net [42]. |
| Dataset Splits | Yes | Without assuming a hard-to-obtain clean validation set, we propose to use a self-validation set, which is instantaneously generated in every AL round. |
| Hardware Specification | Yes | All methods are implemented with Py Torch 1.8.0 and executed on a single NVIDIA Tesla V100 GPU. |
| Software Dependencies | Yes | All methods are implemented with Py Torch 1.8.0 and executed on a single NVIDIA Tesla V100 GPU. |
| Experiment Setup | Yes | The total number r of rounds is set to 10. Following the prior open-set AL setup [13, 16], we set the labeling cost c IN = 1 for IN examples and c OOD = 1 for OOD examples. For the class-split setup, the labeling budget b per round is set to 500 for CIFAR10/100 and 1,000 for Image Net. Regarding the open-set noise ratio τ, we configure four different levels from light to heavy noise in {10%, 20%, 40%, 60%}. For the architecture of MQ-Net, we use a 2-layer MLP with the hidden dimension size of 64 and the Sigmoid activation fuction. |