An Information-Theoretic Framework for Unifying Active Learning Problems
Authors: Quoc Phong Nguyen, Bryan Kian Hsiang Low, Patrick Jaillet9126-9134
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically evaluate the performance of our proposed algorithms using synthetic benchmark functions, a real-world dataset, and in hyperparameter tuning of machine learning models. |
| Researcher Affiliation | Academia | 1Dept. of Computer Science, National University of Singapore, Republic of Singapore 2Dept. of Electrical Engineering and Computer Science, MIT, USA |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available at https://github.com/qphong/bes-mp. |
| Open Datasets | Yes | Details of the synthetic functions are available at https://www.sfu.ca/ssurjano/optimization.html. Fig. 4i shows the results for an LSE problem on an estimated real-world phosphorus field (Webster and Oliver 2007). Firstly, we train a logistic regression model on the MNIST dataset which consists of 28 28 grayscale images of 10 handwritten digits. Secondly, we train a CNN on the CIFAR-10 dataset which consists of 50K 32 32 color images in 10 classes. |
| Dataset Splits | Yes | The objective function is the validation accuracy on a validation set of 14K images. The objective function is the validation accuracy on a validation set of 10K images. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers) needed to replicate the experiment. |
| Experiment Setup | Yes | Each experiment is repeated 30 times to account for the randomness in the observation and the optimization. The GP hyperparameters are learned using maximum likelihood estimation (MLE). The noise variance in the experiments with the synthetic benchmark functions is 0.01. The GP hyperparameters are learned using MLE and |F | is set to 5. The hyperparameters include the L2 regularization weight (in [10 6, 1]), the batch size (in [20, 500]), and the learning rate (in [10 3, 1]). The hyperparameters include the batch size (in [32, 512]), the learning rate (in [10 6, 10 2]) and the learning rate decay (in [10 7, 10 3]) of the RMSprop optimization method, the convolutional filter size (in [128, 256]), and the number of hidden neurons in the dense layer (in [64, 256]). The tolerance α is specified as 0.2. |