Meta Learning Backpropagation And Improving It
Authors: Louis Kirsch, Jürgen Schmidhuber
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | First, we demonstrate the capabilities of the VSML RNN by showing that it can implement neural forward computation and backpropagation in its recurrent dynamics on the MNIST [21] and Fashion MNIST [59] dataset. Then, we show how we can meta learn an LA from scratch on one set of datasets and then successfully apply it to another (out of distribution). |
| Researcher Affiliation | Academia | 1The Swiss AI Lab IDSIA, University of Lugano (USI) & SUPSI, Lugano, Switzerland 2King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia |
| Pseudocode | Yes | Algorithm 1 VSML: Meta Training |
| Open Source Code | No | The paper mentions using "weights & biases [5] for their great experiment tracking software and support" but does not state that the authors' own code for the described methodology is publicly available or provide a link. |
| Open Datasets | Yes | First, we demonstrate the capabilities of the VSML RNN by showing that it can implement neural forward computation and backpropagation in its recurrent dynamics on the MNIST [21] and Fashion MNIST [59] dataset. ... (1) Kuzushiji MNIST [7] with 10 classes, (2) EMNIST [9] with 62 classes |
| Dataset Splits | No | The paper describes meta-training and meta-testing phases and mentions random sampling, but it does not specify explicit train/validation/test dataset splits (e.g., percentages or sample counts) for reproducibility. |
| Hardware Specification | Yes | This work was supported by the ERC Advanced Grant (no: 742870) and computational resources by the Swiss National Supercomputing Centre (CSCS, projects s978 and s1041). We also thank NVIDIA Corporation for donating several DGX machines as part of the Pioneers of AI Research Award, IBM for donating a Minsky machine |
| Software Dependencies | No | The paper mentions "weights & biases [5]" as experiment tracking software but does not provide version numbers for any key software components or libraries used in the experiments. |
| Experiment Setup | No | The paper states: "Hyperparameters, training details, and additional experiments can be found in the appendix." Since these details are explicitly stated to be in the appendix and not in the main text, they do not meet the criteria of being present in the main body of the paper. |