A shooting formulation of deep learning
Authors: François-Xavier Vialard, Roland Kwitt, Susan Wei, Marc Niethammer
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that our particle-ensemble shooting formulation can achieve competitive performance. Finally, though the current work is inspired by continuous-depth neural networks, the particle-ensemble shooting formulation also applies to discrete-time networks and may lead to a new fertile area of research in deep learning parameterization.4 Experiments Our goal is to demonstrate that it is possible to learn DNNs by optimizing only over the initial conditions of critical networks. |
| Researcher Affiliation | Academia | François-Xavier Vialard LIGM, Univ. Gustave Eiffel, CNRS francois-xavier.vialard@u-pem.fr Roland Kwitt Department of Computer Science University of Salzburg Roland.Kwitt@sbg.ac.at Susan Wei School of Mathematics and Statistics University of Melbourne susan.wei@unimelb.edu.au Marc Niethammer Department of Computer Science University of North Carolina at Chapel Hill mn@cs.unc.edu |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Source code is available at: https://github.com/uncbiag/neuro_shooting |
| Open Datasets | Yes | Next, we revisit the spiral ODE example of [13] following the nonlinear dynamics x = Ax3, x R2 (where the power is component-wise).To study the impact of the inflation factor α in a classification regime, we replicate the concentric circles setting of [14].Here, we are given sequences of a rotating MNIST digit (along 16 angles, linearly spaced in [0, 2π]).Finally, we replicate the bouncing balls experiment of [40]. |
| Dataset Splits | Yes | One fixed time point is consistently left-out and later evaluated during testing. We use the same convolutional autoencoder of [40] with the Up Down model operating in the internal representation space after the encoder. [...] randomly drop four time points of each sequence during training.For model selection, we rely on the provided validation set. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, processor types, or memory amounts used for running the experiments. |
| Software Dependencies | No | The paper mentions common deep learning concepts and uses (e.g., 'autoencoder'), but it does not specify any software names with version numbers, such as Python versions, PyTorch versions, or specific library versions. |
| Experiment Setup | Yes | All experiments use the Up Down model with quadratic penalty function R. Detailed experimental settings, including weights for the quadratic penalty function, can be found in Appendix F. We use 15 particles for our experiments. We use an L2 norm loss (calculated on all intermediate time-points) and 25 particles. We use the same convolutional autoencoder of [40] with the Up Down model operating in the internal representation space after the encoder. Our Up Down (dynamic with particles) model uses 100 particles. |