Learning Algorithms for Active Learning
Authors: Philip Bachman, Alessandro Sordoni, Adam Trischler
ICML 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using the Omniglot and Movie Lens datasets, we test our model in synthetic and practical settings. |
| Researcher Affiliation | Industry | 1Microsoft Maluuba, Montreal, Canada. Correspondence to: P. Bachman <phbachma@microsoft.com>, A. Sordoni <alsordon@microsoft.com>. |
| Pseudocode | Yes | Algorithm 1 End-to-end active learning loop (for Eq. 3) |
| Open Source Code | No | The paper does not provide concrete access to source code (specific repository link, explicit code release statement, or code in supplementary materials) for the methodology described. |
| Open Datasets | Yes | We evaluate our active learner on Movie Lens-20M, a standard dataset for recommendation tasks. The dataset contains approximately 20M ratings on 27K movies by 138K users. The ratings are on an ordinal 10-point scale, from 0.5 to 5 with intervals of 0.5. We subsample the dataset by selecting 4000 movies and 6000 users with the most ratings. After filtering, the dataset contains approximately 1M ratings. We partition the data randomly into 5000 training users and 1000 test users. The training set represents the users already in the system who are used to fit the model parameters. We use the test users to evaluate our active learning approach. For each user, we randomly pick 50 ratings to include in the support set (movies that the user can be queried about) and 10 movies and ratings for the held-out set. We ensure that movies in the held-out set and in the support set do not overlap. |
| Dataset Splits | Yes | For each additional label we tune the regularization parameter by monitoring performance on a separate set of validation episodes. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions architectural components like LSTMs, convnets, and optimizers (ADAM), but does not provide specific version numbers for any software libraries or frameworks (e.g., Python, TensorFlow, PyTorch). |
| Experiment Setup | Yes | For the context-free embedding function we use a three-layer convolutional network. The first two layers use 5 5 convolutions with 64 filters and downsample with a double stride. The third layer uses a 3 3 convolution with 64 filters and no downsampling. These layers produce a 7 7 64 feature map that we flatten and pass through a fully connected layer. All convolutional layers use the leaky Re LU nonlinearity (Maas et al., 2013). We setup N-way, K-shot Omniglot classification as follows. We randomly pick N character classes from the available train/test classes. Then, we build a support set by randomly sampling 5 items for each character class, e.g. in the 5-way setting, there are 25-items in the support set. |