Heavy Ball Neural Ordinary Differential Equations
Authors: Hedi Xia, Vai Suliafu, Hangjie Ji, Tan Nguyen, Andrea Bertozzi, Stanley Osher, Bao Wang
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We verify the advantages of HBNODEs over NODEs on benchmark tasks, including image classification, learning complex dynamics, and sequential modeling. |
| Researcher Affiliation | Academia | Hedi Xia Department of Mathematics University of California, Los Angeles Vai Suliafu Scientific Computing and Imaging (SCI) Institute University of Utah, Salt Lake City, UT, USA Hangjie Ji, Tan M. Nguyen, Andrea L. Bertozzi, and Stanley J. Osher Department of Mathematics University of California, Los Angeles Bao Wang Department of Mathematics Scientific Computing and Imaging (SCI) Institute University of Utah, Salt Lake City, UT, USA |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at https://github.com/hedixia/Heavy Ball NODE. |
| Open Datasets | Yes | We verify the advantages of HBNODEs over NODEs on benchmark tasks, including image classification, learning complex dynamics, and sequential modeling. ... We train NODEs for CIFAR10 classification using the same model and experimental settings as in [10]... We compare the performance of HBNODE and GHBNODE with the existing ODE-based models on MNIST and CIFAR10 classification tasks using the same setting as in [10]. ... to study the vibration of an airplane dataset [33]. ... The dataset [3] consists of a dynamical system from kinematic simulation of a person walking from a pre-trained policy, aiming to learn the kinematic simulation of the Mu Jo Co physics engine [48]. |
| Dataset Splits | Yes | We use the first 50% of data as our train set, the next 25% as validation set, and the rest as test set. |
| Hardware Specification | Yes | All experiments are conducted on a server with 2 NVIDIA Titan Xp GPUs. |
| Software Dependencies | No | The paper mentions using "Adam [25] as the benchmark optimization solver" and "Dormand Prince-45 as the numerical ODE solver" but does not specify their version numbers or other software dependencies with versions. |
| Experiment Setup | Yes | For all the experiments, we use Adam [25] as the benchmark optimization solver (the learning rate and batch size for each experiment are listed in Table 1) and Dormand Prince-45 as the numerical ODE solver. For HBNODE and GHBNODE, we set γ = sigmoid(θ), where θ is a trainable weight initialized as θ = 3. |