Homotopy-based training of NeuralODEs for accurate dynamics discovery
Authors: Joon-Hyuk Ko, Hankyul Koh, Nojun Park, Wonho Jhe
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through benchmark experiments, we demonstrate our method achieves competitive or better training loss while often requiring less than half the number of training epochs compared to other model-agnostic techniques. Furthermore, models trained with our method display better extrapolation capabilities, highlighting the effectiveness of our method. |
| Researcher Affiliation | Academia | Joon-Hyuk Ko Department of Physics & Astronomy Seoul National University Seoul, 08826, South Korea jhko725@snu.ac.kr Hankyul Koh Department of Physics & Astronomy Seoul National University Seoul, 08826, South Korea physics113@snu.ac.kr Nojun Park Department of Physics Massachusetts Institute of Technology MA, 02142, United States bnj11526@mit.edu Wonho Jhe Department of Physics & Astronomy Seoul National University Seoul, 08826, South Korea whjhe@snu.ac.kr |
| Pseudocode | Yes | Algorithm 1 Homotopy-based Neural ODE training |
| Open Source Code | Yes | We provide all of the code for this paper in https://github.com/Jhko725/Neural ODEHomotopy. |
| Open Datasets | Yes | Lotka-Volterra system The Lotka-Volterra system is a simplified model of prey and predator populations [43]. Double pendulum For this system, we used the real-world measurement data from Schmidt & Lipson [59]. Lorenz system The Lorenz system displays highly chaotic behavior, and serves as a stress test for time series prediction tasks. |
| Dataset Splits | No | The paper specifies training data and test/extrapolation data but does not explicitly mention a separate validation dataset or how one might be constructed using specific splits (e.g., percentages or counts) for hyperparameter tuning. While hyperparameter selection is described, it does not detail a distinct validation split. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for experiments, such as GPU models, CPU types, or memory specifications. It only mentions software frameworks and libraries. |
| Software Dependencies | No | Our experiments were implemented in Py Torch [46], using the pytorch-lightning [16] framework. The wandb library [7] was used to keep track of all experiments as well as to perform hyperparameter sweeps. ... integration was performed using the adaptive step size dopri5 solver from the torchdiffeq package [11]. While software names are mentioned, no specific version numbers for PyTorch, Pytorch-Lightning, wandb, or torchdiffeq are provided. This makes the software dependencies not fully reproducible. |
| Experiment Setup | Yes | Our final implementation of the homotopy training algorithm has five hyperparameters, as described below. Coupling strength (k) ... Homotopy parameter decrement ratio (κ) ... Number of homotopy steps (s) ... Epochs per homotopy step (nepoch) ... Learning rate (η). Section F. Hyperparameter selection, with Table 1, 2, 3, 4, provides detailed values for all hyperparameters used for vanilla gradient descent, multiple shooting, and homotopy optimization across different models and datasets. |