Active Learning with Model Selection
Authors: Alnur Ali, Rich Caruana, Ashish Kapoor
AAAI 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically demonstrate on six test problems that this algorithm is nearly as effective as an active learning oracle that knows the optimal model in advance. |
| Researcher Affiliation | Collaboration | Alnur Ali Machine Learning Department Carnegie Mellon University alnurali@cmu.edu; Rich Caruana Microsoft Research rcaruana@microsoft.com; Ashish Kapoor Microsoft Research akapoor@microsoft.com |
| Pseudocode | Yes | Algorithm 1 ALMS. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code or a link to a code repository for the methodology described. |
| Open Datasets | Yes | We experimented with six data sets. The 1st column in Figure 2 shows learning curves for ALMS and the four baseline methods vs. the number of sampled labels (rounds of active learning). Figure 2 and Table 1 show results for datasets such as USPS, IONOSPHERE, MUSHROOM, HEART, VOTE, and PIMA. These are standard benchmark datasets commonly used in machine learning research. |
| Dataset Splits | Yes | The allocation of points to the training and validation sets is made dynamically by the algorithm during active learning. On average, across the six problems, ALMS allocates about one third of labels to validation and about two thirds of labels to training. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments, such as CPU or GPU models. |
| Software Dependencies | No | The paper mentions machine learning models and methods like 'ℓ2 regularized logistic regression', 'RBF SVMs', 'Query by Committee', and 'Platt’s method', but it does not specify any software libraries or dependencies with version numbers. |
| Experiment Setup | Yes | Five of the experiments use ℓ2 regularized logistic regression (regularization parameter λ {2 10, . . . , 210}), and one of the experiments uses RBF SVMs (C {10 2, . . . , 102} γ {10 2, . . . , 102}). All results average over 100 trials. |