A Simple Neural Attentive Meta-Learner
Authors: Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, Pieter Abbeel
ICLR 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In the most extensive set of meta-learning experiments to date, we evaluate the resulting Simple Neural Attent Ive Learner (or SNAIL) on several heavily-benchmarked tasks. On all tasks, in both supervised and reinforcement learning, SNAIL attains state-of-the-art performance by significant margins. |
| Researcher Affiliation | Collaboration | UC Berkeley, Department of Electrical Engineering and Computer Science Embodied Intelligence {nmishra, rohaninejadm, c.xi, pabbeel}@berkeley.edu Accepted as a conference paper at ICLR 2018 Authors contributed equally and are listed in alphabetical order. Part of this work was done at Open AI. |
| Pseudocode | Yes | 1: function DENSEBLOCK(inputs, dilation rate R, number of filters D): 2: xf, xg = Causal Conv(inputs, R, D), Causal Conv(inputs, R, D) 3: activations = tanh(xf) * sigmoid(xg) 4: return concat(inputs, activations) |
| Open Source Code | No | Some video results can be found at https://sites.google.com/view/snail-iclr-2018/. (This link is for video results, not source code.) |
| Open Datasets | Yes | Introduced by Lake et al. (2011), Omniglot consists of black-and-white images of handwritten characters gathered from 50 languages, for a total of 1632 different classes with 20 instances per class. ... Mini-Image Net is a more difficult benchmark; a subset of the well-known Image Net dataset, it consists of 84 84 color images from 100 different classes with 600 instances per class. We used the split released by Ravi & Larochelle (2017) and used by a number of other works |
| Dataset Splits | Yes | Mini-Image Net ... We used the split released by Ravi & Larochelle (2017) and used by a number of other works, with 64 classes for training, 16 for validation, and 20 for testing. |
| Hardware Specification | No | The paper does not specify any particular hardware (e.g., GPU models, CPU types, or cloud computing instances) used for running the experiments. |
| Software Dependencies | No | The paper mentions optimizers like 'Adam (Kingma & Ba, 2015)' and methods like 'trust region policy optimization with generalized advantage estimation (TRPO with GAE; Schulman et al. (2015; 2016))', but does not list specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions or other libraries). |
| Experiment Setup | Yes | The hyperparameters are listed in Table 7. For multi-armed bandits, tabular MDPs, and visual navigation, we used the same hyperparamters as Duan et al. (2016) to make our results directly comparable; additional tuning could potentially improve SNAIL s performance. Table 7: The TRPO + GAE hyperparameters we used in our RL experiments. Hyperparameter: Batch Size (timesteps), Discount, GAE λ, Mean KL |