Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints
Authors: Alistair White, Niki Kilbertus, Maximilian Gelbrecht, Niklas Boers
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In extensive empirical evaluations, we demonstrate that SNDEs outperform existing methods while broadening the types of constraints that can be incorporated into NDE training. |
| Researcher Affiliation | Academia | Alistair White Technical University of Munich Potsdam Institute for Climate Impact Research Niki Kilbertus Technical University of Munich Hemholtz AI, Munich Maximilian Gelbrecht Technical University of Munich Potsdam Institute for Climate Impact Research Niklas Boers Technical University of Munich Potsdam Institute for Climate Impact Research University of Exeter |
| Pseudocode | No | The paper describes the mathematical formulation of the method but does not include a distinct pseudocode block or algorithm. |
| Open Source Code | Yes | All code is publicly available at https://github.com/white-alistair/Stabilized-Neural-Differential-Equations. |
| Open Datasets | No | The paper generates its own trajectories for training, rather than using a pre-existing public dataset. For instance: 'We train on 40 trajectories with initial conditions (x, y, x, y) = (1 e, 0, 0, p 1 e/1+e), where the eccentricity e is sampled uniformly via e U(0.5, 0.7).' |
| Dataset Splits | Yes | Trajectories for training and validation are independent, that is, a given trajectory is used exclusively either for training or validation. Each trajectory is split using a multiple-shooting approach into non-overlapping chunks of 3 timesteps each. |
| Hardware Specification | Yes | All experiments are trained for 1,000 epochs on an Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz. |
| Software Dependencies | Yes | The authors also thank the creators and maintainers of the Julia programming language [13] and the open-source packages Differential Equations.jl [66], Flux.jl [46], Zygote.jl [45], Complexity Measures.jl [41] and Makie.jl [27], all of which were essential in preparing this work. ... Trajectories are generated using the 9(8) explicit Runge-Kutta algorithm due to Verner [74], implemented in Differential Equations.jl [66] as Vern9. |
| Experiment Setup | Yes | All experiments are trained for 1,000 epochs using the Adam W optimizer [56] with weight decay of 10 6 and an exponentially decaying learning rate schedule. ... The stabilization hyperparameter γ as well as network sizes, learning rates, and the number of additional dimensions for ANODEs are optimized for each experiment and are summarized in Table 4. |