Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Reducing Variance in Meta-Learning via Laplace Approximation for Regression Tasks
Authors: Alfredo Reichlin, Gustaf Tegnér, Miguel Vasco, Hang Yin, Mårten Björkman, Danica Kragic
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results demonstrate the effectiveness of the proposed method and highlight the importance of variance reduction in meta-learning. [...] 5 Experiments To begin with, we test the validity of using the Laplace approximation to compute the task-adapted parameters for a simple sine regression problem. Additionally, we show how LAVA exhibits a much lower variance in the posterior estimation in comparison to standard GBML. [...] We further evaluate our proposed model on dynamical systems tasks of varying complexity in regard to the family of functions and dimensionality of the task space as well as regression of two real-world datasets. We compare the results of our model against other GBML models. |
| Researcher Affiliation | Academia | Alfredo Reichlin EMAIL KTH Royal Institute of Technology Gustaf Tegnér EMAIL KTH Royal Institute of Technology Miguel Vasco EMAIL KTH Royal Institute of Technology Hang Yin EMAIL University of Copenhagen Mårten Björkman EMAIL KTH Royal Institute of Technology Danica Kragic EMAIL KTH Royal Institute of Technology |
| Pseudocode | Yes | Algorithm 1 LAVA Pseudo-Code [...] Algorithm 2 Hessian Pseudo-Code in Py Torch. |
| Open Source Code | No | The paper does not provide an explicit statement about the release of source code for the described methodology, nor does it include a direct link to a code repository. It mentions PyTorch as a framework but not their specific implementation code. |
| Open Datasets | Yes | We consider the Beijing Air Quality Dataset (Zhang et al., 2017) which is a time-series dataset containing recordings of air quality across 12 monitoring sites. [...] We make use of the Radio ML 2018.01A dataset described in O Shea et al. (2018). [...] We further experiment with classification on the Mini-Imagenet dataset (Vinyals et al., 2016). |
| Dataset Splits | Yes | For each measurement, a time-variable t is appended, indicating a positional embedding in the time-series. The task is then predicting the air-quality measurement from the given time t. [...] For testing, we withhold 25% of the signals. [...] We use the training-set split as used in Ravi & Larochelle (2017) which leaves 64 classes for training, 16 for validation and 20 for test. We experiment with 5-way classification in either a 1-shot or 5-shot setting. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU, GPU models, memory) used for running the experiments. It mentions 'implemented in PyTorch' and 'automatic differentiation framework' but no hardware specifics. |
| Software Dependencies | No | The paper mentions 'Adam (Kingma & Ba, 2014) optimizer' and 'Py Torch (Paszke et al., 2019)' but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | For all of the experiments and all the baselines, we fix the architecture of the meta-learner fθ to be a multi-layer perceptron with 3 hidden layers of 64 hidden units together with ReLU activations. We use a meta batch size of 10 tasks and train all the models with Adam (Kingma & Ba, 2014) optimizer with a learning rate of 10-3. We use the inner learning rate α = 0.1 for the adaptation step and MSE as the adaptation loss. All experiments were run for 5 different seeds to compute mean and standard deviations. For LLAMA we use η = 10-6, for PLATIPUS we scale the KL loss by 0.1, for BMAML we use 10 particles and use MSE rather than the chaser loss for a fair comparison. [...] In our experiments, we consider ϵ = 0.1. |