Active Bayesian Assessment of Black-Box Classifiers
Authors: Disi Ji, Robert L. Logan, Padhraic Smyth, Mark Steyvers7935-7944
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate significant gains from our proposed active Bayesian approach via a series of systematic empirical experiments assessing the performance of modern neural classifiers (e.g., Res Net and BERT) on several standard image and text classification datasets. |
| Researcher Affiliation | Academia | 1Department of Computer Science, University of California, Irvine 2Department of Cognitive Sciences, University of California, Irvine |
| Pseudocode | Yes | Algorithm 1 Thompson Sampling(p, q, r, M) |
| Open Source Code | Yes | Code and scripts for all of our experiments are available at https://github.com/disiji/ active-assess. |
| Open Datasets | Yes | CIFAR-100 (Krizhevsky and Hinton 2009), SVHN (Netzer et al. 2011) and Image Net (Russakovsky et al. 2015)), and text classification (20 Newsgroups (Lang 1995) and DBpedia (Zhang, Zhao, and Le Cun 2015)). |
| Dataset Splits | No | The paper mentions using 'standard training sets' and assessing on 'test sets', but it does not specify explicit train/validation/test splits (e.g., percentages or exact counts for each split) for the overall datasets used. |
| Hardware Specification | No | The paper does not specify any hardware details such as GPU models, CPU types, or cloud computing instances used for the experiments. |
| Software Dependencies | No | The paper mentions software components like 'Res Net' and 'BERT' but does not provide specific version numbers for any software dependencies used in the experiments. |
| Experiment Setup | Yes | We set the prior strengths as αg + βg = N0 = 2 for Beta priors and P αg = N0 = 1 for Dirichlet priors in all experiments... In our experiments we use ϵ = 0.05 and the cumulative densities µ are estimated with 10,000 Monte Carlo samples. |