Simulation-Free Training of Neural ODEs on Paired Data
Authors: Semin Kim, Jaehoon Yoo, Jinwoo Kim, Yeonwoo Cha, Saehoon Kim, Seunghoon Hong
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the effectiveness of our method on both regression and classification tasks, where our method outperforms existing NODEs with a significantly lower number of function evaluations. The code is available at https://github.com/seminkim/simulation-free-node. |
| Researcher Affiliation | Collaboration | Semin Kim1 Jaehoon Yoo1 Jinwoo Kim1 Yeonwoo Cha1 Saehoon Kim2 Seunghoon Hong1 1KAIST 2Kakao Brain |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available at https://github.com/seminkim/simulation-free-node. |
| Open Datasets | Yes | Following prior work [11, 13], we use MNIST [26], SVHN [33], and CIFAR10 [25] for image classification experiments. For regression tasks, we use UCI regression datasets [21], adhering to the protocol used by CARD. |
| Dataset Splits | Yes | Additionally, we split the training set into a train-validation split with a ratio of 6:4, and utilized the validation metric for early-stopping. |
| Hardware Specification | Yes | We conducted experiments on our internal cluster with two types of machine. We list their specifications below. 1. Intel Xeon Gold 6330 CPU and NVIDIA RTX A6000 GPU (with 48GB VRAM) 2. Intel Xeon Gold 6230 CPU and NVIDIA RTX 3090 GPU (with 24GB VRAM) |
| Software Dependencies | No | The paper mentions using "Adam optimizer [24]", "dopri5 [10] adaptive-step solver implemented in torchdiffeq [6] package", but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | We train all models for 100,000 iterations using the Adam optimizer [24] with a cosine learning rate scheduler. For all classification experiments, we utilize a batch size of 1024. [...] By default, we set the learning rate to 1e-3 for MNIST and 3e-4 for CIFAR10 and SVHN. For our method, we set the ratio of explicitly sampling t = 0 to 10% for all datasets. Regarding the noise introduced to the label autoencoder, we set the standard deviation σ to 3 for MNIST, 7 for SVHN, and 10 for CIFAR10. |