A Metalearned Neural Circuit for Nonparametric Bayesian Inference
Authors: Jake Snell, Gianluca Bencomo, Tom Griffiths
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experimental results show that the metalearned neural circuit achieves comparable or better performance than particle filter-based methods that explicitly perform Bayesian nonparametric inference while being faster and simpler to use. 4 Experiments: We apply the neural circuit on three data settings: a synthetic dataset where the form of the DPMM is known, sequences of labels generated from a CRP on Image Net [51], and sequences sampled directly from the long-tailed i Naturalist 2018 species classification dataset [61]. |
| Researcher Affiliation | Academia | Jake C. Snell Department of Computer Science Princeton University jsnell@princeton.edu Gianluca M. Bencomo Department of Computer Science Princeton University gb5435@princeton.edu Thomas L. Griffiths Department of Computer Science Department of Psychology Princeton University tomg@princeton.edu |
| Pseudocode | No | The paper contains mathematical formulations and descriptions of the model, but no clearly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | Our code is available online 1. 1https://github.com/jakesnell/neural-circuits |
| Open Datasets | Yes | We downloaded the weights of a pretrained Res Net-18 from TIMM [66] and extracted the 512dimensional penultimate layer activations from the entire ILSVRC 2012 dataset [51]. open-set classification on the long-tailed i Naturalist 2018 dataset [61]. |
| Dataset Splits | Yes | Third, we use an Image Net class split of 350 meta-train, 350 meta-validation, and 300 meta-test in order to mitigate the potential risk of overfitting. For the i Naturalist 2018 results, overfitting was monitored on a separate meta-validation class split. |
| Hardware Specification | Yes | Experiments were performed using NVIDIA A100 GPUs with 40 GB of GPU memory and 64 GB of CPU memory across 4 threads. |
| Software Dependencies | No | All methods are implemented in Py Torch [4] and are GPU-enabled. The clustering metrics were computed using the adjusted_rand_score and adjusted_mutual_info_score functions from Scikit Learn [5]. |
| Experiment Setup | Yes | For all of our experiments neural circuit RNN cell was chosen to be a 2-layer gated recurrent unit (GRU) [1] with hidden size 1024. The maximum number of output logits was set to be equal to the sequence length used for metalearning, as this is the maximum number of classes that could possibly be encountered. Training was performed over 10,000 minibatches each of size 128 sequences of length 100 (256 sequences of length 500 for the i Naturalist experiments). The CRP coefficient was set to α = 1.0 for the synthetic and Image Net-CRP experiments, and was set to a range of different values {1, 2, 5, 10, 20, 50, 100, 200} in the i Naturalist experiments. The neural circuit was trained using Adam with learning rate 0.001 and the particle filter using learning rate 0.1. |