Dissecting Neural ODEs
Authors: Stefano Massaroli, Michael Poli, Jinkyoo Park, Atsushi Yamashita, Hajime Asama
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The results for five experiments are reported in Table 4. IL NODEs consistently preserve lower NFEs than other variants, whereas second order Neural ODEs offer a parameter efficient alternative. The performance gap widens on CIFAR10, where the disadvantage of fixed 0 initial conditions forces 0 augmented Neural ODEs into performing a high number of function evaluations. Table 1: Mean test results across 10 runs on MNIST and CIFAR. We report the mean NFE at convergence. Input layer and higher order augmentation improve task performance and preserve low NFEs at convergence. |
| Researcher Affiliation | Academia | Stefano Massaroli The University of Tokyo, Diff Eq ML massaroli@robot.t.u-tokyo.ac.jp Michael Poli KAIST, Diff Eq ML poli_m@kaist.ac.kr Jinkyoo Park KAIST jinkyoo.park@kaist.ac.kr Atsushi Yamashita The University of Tokyo yamashita@robot.t.u-tokyo.ac.jp Hajime Asama The University of Tokyo asama@robot.t.u-tokyo.ac.jp |
| Pseudocode | No | The paper includes mathematical formulations and derivations but no structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code to reproduce all the experiments present in the paper is built on Torch Dyn (Poli et al., 2020b) and Py Torch Lighning (Falcon et al., 2019) libraries, can be found in the following repo: https://github.com/Diff Eq ML/ diffeqml-research/tree/master/dissecting-neural-odes. |
| Open Datasets | Yes | Table 1: Mean test results across 10 runs on MNIST and CIFAR. We report the mean NFE at convergence. Input layer and higher order augmentation improve task performance and preserve low NFEs at convergence. The concentric annuli (Dupont et al., 2019) dataset is utilized, and the models are qualitatively evaluated based on the complexity of the learned flows and on how accurately they extrapolate to unseen points, i.e. the learned decision boundaries. |
| Dataset Splits | No | The paper mentions training and testing on datasets like MNIST and CIFAR, but does not provide specific details on the dataset splits (e.g., percentages for train/validation/test sets). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | Yes | The code to reproduce all the experiments present in the paper is built on Torch Dyn (Poli et al., 2020b) and Py Torch Lighning (Falcon et al., 2019) libraries, can be found in the following repo: https://github.com/Diff Eq ML/ diffeqml-research/tree/master/dissecting-neural-odes. |
| Experiment Setup | No | Main objective of these experiments is to rank the efficieny of different augmentation strategies; for this reason, the setup does not involve hybrid or composite Neural ODE architectures and data augmentation. The input network hx is composed of a single, linear layer. However, this is not detailed enough for specific hyperparameters like learning rate, batch size, number of epochs, or optimizer choice. |