BayesPCN: A Continually Learnable Predictive Coding Associative Memory
Authors: Jinsoo Yoo, Frank Wood
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that Bayes PCN can recall corrupted i.i.d. high-dimensional data observed hundreds to a thousand timesteps ago without a large drop in recall ability compared to the state-of-the-art offline-learned parametric memory models. This section assesses Bayes PCN s recall capabilities on image recovery tasks and examines the effect of forget on Bayes PCN s behaviour. We used CIFAR10 [Krizhevsky et al., 2009] and Tiny Image Net [Le and Yang, 2015] datasets for all experiments... |
| Researcher Affiliation | Academia | Jason Yoo Department of Computer Science University of British Columbia Vancouver, Canada jasony97@cs.ubc.ca Frank Wood Department of Computer Science University of British Columbia Vancouver, Canada fwood@cs.ubc.ca |
| Pseudocode | Yes | Algorithm 1 Memory Write and Algorithm 2 Memory Forget |
| Open Source Code | Yes | 1Code is available at https://github.com/plai-group/bayes-pcn |
| Open Datasets | Yes | We used CIFAR10 [Krizhevsky et al., 2009] and Tiny Image Net [Le and Yang, 2015] datasets for all experiments, whose images are of size 3 × 32 × 32 and 3 × 64 × 64 respectively. |
| Dataset Splits | No | The paper describes using CIFAR10 and Tiny Image Net for sequential, continual learning, stating 'Bayes PCN models were given one image to store into their memories per timestep for up to 1024 images'. However, it does not explicitly specify traditional training, validation, or test dataset splits (e.g., percentages, counts, or references to predefined splits) for reproducibility of data partitioning. It focuses on the sequential learning process rather than static dataset splits. |
| Hardware Specification | No | The paper acknowledges support from computational resources such as West Grid, Compute Canada, and Advanced Research Computing at the University of British Columbia (arc.ubc.ca). However, it does not provide specific hardware details such as exact GPU or CPU models, processor types, or memory amounts used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers (e.g., Python 3.8, PyTorch 1.9, CUDA 11.1) needed to replicate the experiment. It only mentions using 'our own autodiff implementation of MHNs'. |
| Experiment Setup | Yes | We experimented with 4-layer Bayes PCN models that have hidden layer sizes of 256, 512, and 1024, particle counts of 1 and 4, and activation functions Re LU and GELU. (from main text) For all GPCN models, we used the Adam optimizer with cosine annealing learning rate schedule (initial learning rate 0.001, warm-up for 500 steps, min learning rate 0.0001, weight decay 0.0001), 1024 epochs, and batch size 64. (from Appendix F) |