Meta-Learning through Hebbian Plasticity in Random Networks
Authors: Elias Najarro, Sebastian Risi
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate our approach on several reinforcement learning tasks with different sensory modalities and more than 450K trainable plasticity parameters. We find that starting from completely random weights, the discovered Hebbian rules enable an agent to navigate a dynamical 2D-pixel environment; likewise they allow a simulated 3D quadrupedal robot to learn how to walk while adapting to morphological damage not seen during training and in the absence of any explicit reward or error signal in less than 100 timesteps. |
| Researcher Affiliation | Academia | Elias Najarro and Sebastian Risi IT University of Copenhagen 2300 Copenhagen, Denmark enaj@itu.dk, sebr@itu.dk |
| Pseudocode | No | The paper describes the steps of the approach and optimization algorithm in narrative text, but it does not include any formally structured pseudocode or algorithm blocks (e.g., labeled 'Algorithm 1'). |
| Open Source Code | Yes | Code is available at https://github.com/enajx/Hebbian Meta Learning. [...] All the code necessary to evolve both the Hebbian networks as well as the static networks with the ES algorithm is available at https://github.com/enajx/Hebbian Meta Learning. |
| Open Datasets | Yes | As a vision-based environment, we use the Car Racing-v0 domain [51], build with the Box2D physics engine. [...] For the state-vector environment we use the open-source Bullet physics engine and its py Bullet python wrapper [57] that includes the Ant robot. |
| Dataset Splits | No | The paper describes how different morphologies are used for training and testing generalization (e.g., 'The third morphology (damaged on left front leg) is left out of training loop in order to subsequently evaluate the generalisation of the networks.'), but it does not specify explicit training/validation/test splits with percentages or sample counts for a single dataset. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU or CPU models, memory, or cloud computing instance specifications used for running the experiments. It only mentions 'typical ES settings'. |
| Software Dependencies | Yes | For the state-vector environment we use the open-source Bullet physics engine and its py Bullet python wrapper [57] that includes the Ant robot [...]. [57] Erwin Coumans and Yunfei Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016 2019. |
| Experiment Setup | Yes | The parameters used for the ES algorithm to optimize both the Hebbian and static networks are the following: a population size 200 for the Car Racing-v0 domain and size 500 for the quadruped, reflecting the higher complexity of this domain. Other parameters were the same for both domains and reflect typical ES settings (ES algorithms are typically more robust to different hyperparameters than other RL approaches [44]), with a learning rate α=0.2, α decay=0.995, σ=0.1, and σ decay=0.999. |