Neural Circuit Architectural Priors for Embodied Control
Authors: Nikhil Bhattasali, Anthony M Zador, Tatiana Engel
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | On a locomotion task, our architecture achieves good initial performance and asymptotic performance comparable with MLPs, while dramatically improving data efficiency and requiring orders of magnitude fewer parameters. Our architecture is interpretable and transfers to new body designs. An ablation analysis shows that constrained excitation/inhibition is crucial for learning, while weight initialization contributes to good initial performance. |
| Researcher Affiliation | Academia | Neuro AI Program, Cold Spring Harbor Laboratory {bhattas,zador,engel}@cshl.edu |
| Pseudocode | No | The paper describes the architecture and equations but does not present structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code and videos are available at: https://sites.google.com/view/ncap-swimmer |
| Open Datasets | Yes | Specifically, we translate C. elegans locomotion circuits into an ANN model controlling a simulated Swimmer agent selected from a standard AI benchmark [Tassa et al., 2020]. and We implement the Swimmer using the standard N = 5 body in the Deep Mind Control Suite [Tassa et al., 2020] built upon the Mu Jo Co physics simulator [Todorov et al., 2012]. |
| Dataset Splits | No | The paper describes training and evaluation within a reinforcement learning environment (Deep Mind Control Suite) and mentions 'timesteps' for training duration and '10 random seeds' for averaging, but does not provide explicit 'training/test/validation dataset splits' in terms of percentages or sample counts for fixed datasets as is common in supervised learning. The concept of dataset splits is not directly applicable in the same way to an RL environment. |
| Hardware Specification | No | Training was performed on a high performance computing cluster running the Linux Ubuntu operating system. RL algorithm training runs were parallelized over 8 cores, while ES algorithm runs were parallelized over 32 cores. |
| Software Dependencies | No | Neural networks were implemented in Py Torch (BSD license) [Paszke et al., 2019]. The RL algorithms were implemented using Tonic (MIT license) [Pardo, 2021]. The ES algorithm was implemented using ES Torch (MIT license) [Karakasli, 2020]. |
| Experiment Setup | Yes | RL Algorithms Standard hyperparameters for PPO and DDPG in Tonic [Pardo, 2021] at commit 48a7b72; timesteps, 5e6. ES Algorithm Population size, 256; noise standard deviation σ, 0.02; L2 weight decay, 0.005; optimized, Adam; learning rate, 0.01; timesteps, 5e7. NCAP Swimmer Oscillator, square wave, period 60 timesteps, width 30 timesteps. |