Meta-Learning Probabilistic Inference for Prediction
Authors: Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, Richard Turner
ICLR 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 EXPERIMENTS AND RESULTS We evaluate VERSA on several few-shot learning tasks. We begin with toy experiments to investigate the properties of the amortized posterior inference achieved by VERSA. We then report few-shot classification results using the Omniglot and mini Image Net datasets in Section 5.2, and demonstrate VERSA s ability to retain high accuracy as the shot and way are varied at test time. In Section 5.3, we examine VERSA s performance on a one-shot view reconstruction task with Shape Net objects. |
| Researcher Affiliation | Collaboration | Jonathan Gordon , John Bronskill University of Cambridge {jg801,jfb54}@cam.ac.uk Matthias Bauer University of Cambridge Max Planck Institute for Intelligent Systems bauer@tue.mpg.de Sebastian Nowozin Google AI Berlin nowozin@google.com Richard E. Turner University of Cambridge Microsoft Research ret26@cam.ac.uk |
| Pseudocode | No | The paper provides computational flow diagrams (Figure 2, Figure 3) but does not include formal pseudocode blocks or sections explicitly labeled 'Algorithm'. |
| Open Source Code | Yes | Source code for the experiments is available at https://github.com/Gordonjo/versa. |
| Open Datasets | Yes | We evaluate VERSA on standard few-shot classification tasks. Specifically, we consider the Omniglot (Lake et al., 2011) and mini Image Net (Ravi and Larochelle, 2017) datasets... Shape Net Core v2 (Chang et al., 2015) is a database of 3D objects... |
| Dataset Splits | Yes | The training, validation, and test sets consist of a random split of 1100, 100, and 423 characters, respectively. (Omniglot Section D.1) ... we use the 64 training, 16 validation, and 20 test class splits defined by (Ravi and Larochelle, 2017). (mini Image Net Section D.2) ... we use 70% of the objects (25,975 in total) for training, 10% for validation (3,710 in total) , and 20% (7423 in total) for testing. (ShapeNet Section E.1) |
| Hardware Specification | Yes | The time taken to evaluate 1000 test tasks with a 5-way, 5-shot mini Image Net trained model using MAML (https://github.com/cbfinn/maml) is 302.9 seconds whereas VERSA took 53.5 seconds on a NVIDIA Tesla P100-PCIE-16GB GPU. |
| Software Dependencies | No | The paper mentions using the Adam optimizer (Kingma and Ba, 2015) but does not specify version numbers for any software libraries or dependencies (e.g., Python, TensorFlow, PyTorch versions). |
| Experiment Setup | Yes | We use the Adam (Kingma and Ba, 2015) optimizer with a constant learning rate of 0.0001 with 16 tasks per batch to train all models. The 5-way 5-shot and 5-way 1-shot models are trained for 80,000 iterations while the 20-way 5-shot model is trained for 60,000 iterations, and the 20-way 1-shot model is trained for 100,000 iterations. In addition, we use a Gaussian form for q and set the number of ψ samples to L = 10. (Section D.1) |