Energy-based learning algorithms for analog computing: a comparative study
Authors: Benjamin Scellier, Maxence Ernoult, Jack Kendall, Suhas Kumar
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we carry out a comparison of seven learning algorithms, namely CL and different variants of EP and Cp L depending on the signs of the perturbations. Specifically, using these learning algorithms, we train deep convolutional Hopfield networks (DCHNs) on five vision tasks (MNIST, F-MNIST, SVHN, CIFAR-10 and CIFAR-100). We find that, while all algorithms yield comparable performance on MNIST, important differences in performance arise as the difficulty of the task increases. |
| Researcher Affiliation | Industry | Benjamin Scellier Rain AI benjamin@rain.ai Maxence Ernoult Rain AI maxence@rain.ai Jack Kendall Rain AI jack@rain.ai Suhas Kumar Rain AI suhas@rain.ai |
| Pseudocode | No | The paper describes the energy-based learning algorithms and procedures in narrative text and mathematical equations, but it does not contain a dedicated pseudocode or algorithm block. |
| Open Source Code | Yes | The code is available at https://github.com/rain-neuromorphics/energy-based-learning |
| Open Datasets | Yes | We train a DCHN on MNIST, Fashion-MNIST, SVHN, CIFAR-10 and CIFAR-100 using each of the seven EBL algorithms. [...] The MNIST dataset is composed of images of handwritten digits [Le Cun et al., 1998]. [...] The CIFAR-10 dataset [Krizhevsky, Hinton, et al., 2009] consists of 60,000 colour images of 32x32 pixels. |
| Dataset Splits | No | The paper details training and test set sizes for each dataset (e.g., "The dataset contains 60,000 training images and 10,000 test images" for MNIST, and "The training set consists of 50,000 images and the test set of 10,000 images" for CIFAR-10), but does not explicitly provide information on validation dataset splits or how a validation set was used. |
| Hardware Specification | Yes | Each run is performed on a single A100 GPU. [...] For example, a full run of 100 epochs on CIFAR-10 yields a test error rate of 10.4% and takes 3 hours 18 minutes on a single A100 GPU |
| Software Dependencies | Yes | The code for the simulations uses PyTorch 1.13.1 and TorchVision 0.14.1. Paszke et al. [2017]. |
| Experiment Setup | Yes | Table 5 contains the architectural details of the network, as well as the hyperparameters used to obtain the results presented in Table 1, Table 2 and Table 6. [...] nudging (β) 0.25 num. iterations at inference (T) 60 num. iterations at training (K) 15 [...] lr conv-weight 1 & bias 1 (η1) 0.0625 momentum 0.9 weight decay (1e-4) 3.0 mini-batch size 128 number of epochs 100 Tmax 100 ηmin (1e-6) 2.0 |