Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Beam Search Optimized Batch Bayesian Active Learning
Authors: Jingyu Sun, Hongjie Zhai, Osamu Saisho, Susumu Takeuchi
AAAI 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We report results on three benchmark datasets. Experimental results show that our method consistently outperforms baseline methods including BALD and Batch BALD in terms of both model quality metrics and acceptable computational cost. |
| Researcher Affiliation | Industry | 1 NTT Computer and Data Science Laboratories 2 NTT Software Innovation Center 3 NTT Social Informatics Laboratories EMAIL, EMAIL, EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1: Greedy algorithm for batch acquisition; Algorithm 2: Beam search with self-adapted constraint for batch acquisition in active learning |
| Open Source Code | No | The paper does not provide any links to open-source code or explicitly state that the code is publicly available. |
| Open Datasets | Yes | The datasets, MNIST( (Le Cun et al. 1998)), Re MNIST( (Kirsch, Van Amersfoort, and Gal 2019)), and CIFAR-10, are used in our experiments. |
| Dataset Splits | Yes | We use a validation set of 2000 samples, a balanced test set of 20,000, and randomly select 20 samples for initial training set (2 per class). |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for the experiments, such as GPU/CPU models or cloud instance types. |
| Software Dependencies | No | The paper mentions 'Adam optimizer' and 'MC dropout' but does not specify versions for any libraries, frameworks, or other software components. |
| Experiment Setup | Yes | We use a convolutional neural network (CNN) consisting of two blocks of convolution (with 32 and 64 5 5 filters), dropout, max pooling, and Re LU. Following these two blocks, there are a fully connected layer of 128 hidden units and MC dropout. All dropouts are with a probability of 0.5. All models are optimized with the Adam optimizer with a learning rate of 0.001 and betas(0.9, 0.999). The initial samples are randomly selected but of the same number per class. We also stop early after three epochs with declining accuracy on validation set for avoiding overfitting. Each experiment is repeated five times with different seeds and different initial samples. Median of these five trials and the lower/upper quartiles are used to draw the accuracy figures. All beam searches are conducted with a width of 32. |