Deep Active Learning with Noise Stability
Authors: Xingjian Li, Pengkun Yang, Yangcheng Gu, Xueying Zhan, Tianyang Wang, Min Xu, Chengzhong Xu
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our method on multiple tasks, including (1) image classification on MNIST (Le Cun et al. 1998), Cifar10 (Krizhevsky, Hinton et al. 2009), SVHN (Netzer et al. 2011), Cifar100 (Krizhevsky, Hinton et al. 2009), and Caltech101 (Fei-Fei, Fergus, and Perona 2006); (2) regression on the Ames Housing dataset (De Cock 2011); (3) semantic segmentation on Cityscapes (Cordts et al. 2016); and (4) natural language processing on MRPC (Dolan and Brockett 2005). Our method is shown to exceed or be comparable to the state-of-the-art baselines on all these tasks. |
| Researcher Affiliation | Academia | Xingjian Li1*, Pengkun Yang2*, Yangcheng Gu3, Xueying Zhan1, Tianyang Wang4, Min Xu1 , Chengzhong Xu5 1Computational Biology Department, Carnegie Mellon University 2Center for Statistical Science, Tsinghua University 3School of Software, Tsinghua University 4Department of Computer Science, University of Alabama at Birmingham 5State Key Lab of IOTSC, University of Macau |
| Pseudocode | Yes | Algorithm 1: Active learning with Noise Stability. Input : T : random initialized neural network, U: unlabeled pool of training data, L: labeled pool of training data; Output : L: updated labeled pool; |
| Open Source Code | No | We conduct all the experiments using Pytorch (Paszke et al. 2017) and our source code will be publicly available. |
| Open Datasets | Yes | We use three classification datasets for evaluations, including MNIST (Le Cun et al. 1998), Cifar10 (Krizhevsky, Hinton et al. 2009) and SVHN (Netzer et al. 2011). |
| Dataset Splits | No | The Cifar10 dataset includes 50000 training and 10000 testing samples, uniformly distributed across 10 classes. SVHN includes 73257 training and 26032 testing samples, and we do not use the additional training data in SVHN, following the common practice. The paper mentions training and testing splits for datasets but does not explicitly state a validation split or how it's handled for reproduction. |
| Hardware Specification | No | The paper does not specify the exact hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | We conduct all the experiments using Pytorch (Paszke et al. 2017) and our source code will be publicly available. The paper mentions PyTorch but does not provide specific version numbers for software dependencies. |
| Experiment Setup | Yes | For our method, we use the following hyper-parameters in all the experiments: K = 30 and ΞΆ = 10 3, to ensure the constraint on noise magnitude. The number of Monte-Carlo sampling for BALD is 50. Other hyper-parameters for baseline methods are set as suggested in the original papers. The network and training details are described in Appendix A.4.1. |