Towards Biologically Plausible Convolutional Networks
Authors: Roman Pogodin, Yash Mehta, Timothy Lillicrap, Peter E Latham
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We split our experiments into two parts: small-scale ones with CIFAR10, CIFAR100 [35] and Tiny Image Net [36], and large-scale ones with Image Net [37]. The former illustrates the effects of data augmentation and dynamic weight sharing on the performance of locally connected networks; the latter concentrates on dynamic weight sharing, as extensive data augmentations are too computationally expensive for large networks and datasets. |
| Researcher Affiliation | Collaboration | Roman Pogodin Gatsby Unit, UCL roman.pogodin.17@ucl.ac.uk Yash Mehta Gatsby Unit, UCL y.mehta@ucl.ac.uk Timothy P. Lillicrap Deep Mind; Co MPLEX, UCL countzero@google.com Peter E. Latham Gatsby Unit, UCL pel@gatsby.ucl.ac.uk |
| Pseudocode | No | No section explicitly labeled "Pseudocode" or "Algorithm" was found, nor were there structured pseudocode blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/romanpogodin/towards-bio-plausible-conv (Py Torch [39] implementation). |
| Open Datasets | Yes | Datasets. CIFAR10 consists of 50k training and 10k test images of size 32 32, divided into 10 classes. CIFAR100 has the same structure, but with 100 classes. For both, we tune hyperparameters with a 45k/5k train/validation split, and train final networks on the full 50k training set. Tiny Image Net consists of 100k training and 10k validation images of size 64 64, divided into 200 classes. As the test labels are not publicly available, we divided the training set into 90k/10k train/validation split, and used the 10k official validation set as test data. Image Net consists of 1.281 million training images and 50k test images of different sizes, reshaped to 256 pixels in the smallest dimension. As in the case for Tiny Image Net, we used the train set for a 1.271 million/10k train/validation split, and 50k official validation set as test data. |
| Dataset Splits | Yes | For both, we tune hyperparameters with a 45k/5k train/validation split, and train final networks on the full 50k training set. Tiny Image Net consists of 100k training and 10k validation images of size 64 64, divided into 200 classes. As the test labels are not publicly available, we divided the training set into 90k/10k train/validation split, and used the 10k official validation set as test data. Image Net consists of 1.281 million training images and 50k test images of different sizes, reshaped to 256 pixels in the smallest dimension. As in the case for Tiny Image Net, we used the train set for a 1.271 million/10k train/validation split, and 50k official validation set as test data. |
| Hardware Specification | Yes | We ran the experiments on our local laboratory cluster, which consists mostly of NVIDIA GTX1080 and RTX5000 GPUs. |
| Software Dependencies | No | The paper mentions PyTorch [39] implementation in Section 5 and Adam W [38] optimizer, but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | We used the Adam W [38] optimizer in all runs. As our dynamic weight sharing procedure always converges to a nearly convolutional solution (see Section 4), we set the weights to the mean directly (within each grid) to speed up experiments. |