Transferring Knowledge across Learning Processes
Authors: Sebastian Flennerhag, Pablo G. Moreno, Neil D. Lawrence, Andreas Damianou
ICLR 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We consider three experiments with increasingly complex knowledge transfer. We measure transfer learning in terms of final performance and speed of convergence, where the latter is defined as the area under the training error curve. We compare Leap to competing meta-learning methods on the Omniglot dataset by transferring knowledge across alphabets (section 4.1). We study Leap s ability to transfer knowledge over more complex and diverse tasks in a Multi-CV experiment (section 4.2) and finally evaluate Leap on in a demanding reinforcement environment (section 4.3). |
| Researcher Affiliation | Collaboration | Sebastian Flennerhag The Alan Turing Institute London, UK sflennerhag@turing.ac.uk Pablo G. Moreno Amazon Cambridge, UK morepabl@amazon.com Neil D. Lawrence Amazon Cambridge, UK lawrennd@amazon.com Andreas Damianou Amazon Cambridge, UK damianou@amazon.com |
| Pseudocode | Yes | Algorithm 1 Leap: Transferring Knowledge over Learning Processes |
| Open Source Code | No | The paper does not provide any explicit statements about releasing source code or links to a code repository for the described methodology. |
| Open Datasets | Yes | The Omniglot (Lake et al., 2015) dataset consists of 50 alphabets... The Multi-CV experiment is more challenging... We apply it in a reinforcement learning environment, specifically Atari 2600 games (Bellemare et al., 2013). |
| Dataset Splits | Yes | For each character in an alphabet, we hold out 5 samples in order to create a task validation set. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware (e.g., GPU models, CPU types) used for running its experiments. It only mentions general setup like 'trained Leap for 100 steps, equivalent to training 1600 agents for 5 million steps' without hardware details. |
| Software Dependencies | No | The paper mentions optimizers like 'Adam (Kingma & Ba, 2015)' and 'RMSProp' with some parameters (ϵ = 10 4, α = 0.99), but it does not specify any software dependencies with version numbers (e.g., TensorFlow, PyTorch, scikit-learn with their respective versions). |
| Experiment Setup | Yes | Table 3: Summary of hyper-parameters for Omniglot. Meta refers to the outer training loop, task refers to the inner training loop. For instance, 'Learning rate 0.1', 'Training steps 1000', 'Batch size (tasks) 20'. In the Atari experiment details, it mentions 'batch size of 32', 'unroll length of 5' and 'RMSProp (using ϵ = 10 4, α = 0.99) with a learning rate of 10 4'. |