H-Mem: Harnessing synaptic plasticity with Hebbian Memory Networks
Authors: Thomas Limbacher, Robert Legenstein
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that the network can be optimized to utilize the Hebbian plasticity processes for its computations. H-Mems can one-shot memorize associations between stimulus pairs and use these associations for decisions later on. Furthermore, they can solve demanding question-answering tasks on synthetic stories. We conducted experiments on a task that requires to form associations between entities that appear together in a sequence of images. We trained a feed-forward H-Mem network on this task. We evaluate the model on the b Ab I data set. |
| Researcher Affiliation | Academia | Thomas Limbacher and Robert Legenstein Institute of Theoretical Computer Science Graz University of Technology 8010 Graz, Austria |
| Pseudocode | No | The paper includes mathematical equations (Eq. 1-6) describing the model's operation but does not present them within a clearly labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | Yes | The model was implemented in TensorFlow (the code it available at https://github.com/IGITUGraz/HMem). |
| Open Datasets | Yes | We randomly chose three classes out of the 10 classes of the CIFAR-10 data set... handwritten digit (from the MNIST data set). We evaluated the model on the b Ab I data set [9]. |
| Dataset Splits | Yes | The networks were trained for 100 epochs on 10 000 examples per task with a batch size of 128 (see Supplementary for details to the model and the training setup). Similar to previous work [11], [13], we performed three independent runs with different random initializations and report the results of the model with the highest validation accuracy in these runs. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions 'implemented in TensorFlow' and 'Adam optimizer [31]', but it does not specify version numbers for TensorFlow or any other software libraries or dependencies used, which would be necessary for full reproducibility. |
| Experiment Setup | Yes | We trained a feed-forward H-Mem network on this task... for 100 epochs with a batch size of 32. We trained an H-Mem network with an embedding dimension d of 80 and a hetero-associative memory with a square association matrix of order m = 100. We used the recurrent model with the number of memory queries N set to 3. The networks were trained for 100 epochs on 10 000 examples per task with a batch size of 128. The weights W s key, W s val, W q key, and Wout are learned during training by minimizing the cross-entropy loss between ˆa and the true answer a using the Adam optimizer [31]. The associative memory matrix W assoc is initialized for each input sequence xt with all its values set to zeros. We performed three independent runs with different random initializations and report the results of the model with the highest validation accuracy in these runs. |