Accurate Bayesian Meta-Learning by Accurate Task Posterior Inference
Authors: Michael Volpp, Philipp Dahlinger, Philipp Becker, Christian Daniel, Gerhard Neumann
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical evaluation aims to study the effect on the predictive performance of (i) our improved TRNG-VI approach as well as of (ii) expressive variational GMM TP approximations in NP-based BML, in (iii) comparison to the state-of-the-art on (iv) a range of practically relevant meta-learning tasks. |
| Researcher Affiliation | Collaboration | 1Karlsruhe Institute of Technology, Karlsruhe, Germany 2Bosch Center for Artificial Intelligence, Renningen, Germany |
| Pseudocode | Yes | A.1.3 ALGORITHM SUMMARY We provide pseudocode for the meta-training stage of our GMM-NP algorithm in Alg. 1 and for the prediction stage in Alg. 2. |
| Open Source Code | Yes | We also provide source code for our proposed GMM-NP algorithm: Source code four our GMM-NP algorithm: https://github.com/ALRhub/gmm_np |
| Open Datasets | Yes | 2D MNIST Image Completion. We use the MNIST handwritten image database (Le Cun & Cortes, 2010). |
| Dataset Splits | No | The paper uses 'unseen test tasks' which implies a split, but does not explicitly provide percentages or counts for training/validation/test splits of the overall meta-datasets, nor does it define a separate validation set. |
| Hardware Specification | No | This work was performed on the Hore Ka supercomputer funded by the Ministry of Science, Research and the Arts Baden-W urttemberg and by the Federal Ministry of Education and Research. The authors further acknowledge support by the state of Baden-W urttemberg through bw HPC. |
| Software Dependencies | No | The paper cites TensorFlow and PyTorch as software used for automatic differentiation and mentions wandb for hyperparameter search, but does not provide specific version numbers for these or any other key software dependencies used in their experiments. |
| Experiment Setup | Yes | For a fair comparison, we employ a fixed experimental protocol for all datasets and models: we first perform a Bayesian hyperparameter search (HPO) to determine optimal algorithm settings, individually for each model-dataset combination. We then retrain the best model with 8 different random seeds and report the median log marginal predictive likelihood (LMLHD) as well as the median mean squared error (MSE), both in dependence of the context set size. (A.3.1 covers details such as DNN architectures, latent dimensionalities, number of GMM components, learning rates and trust region bounds with specific ranges and settings). |