Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series
Authors: Abdul Fatir Ansari, Alvin Heng, Andre Lim, Harold Soh
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results on multiple benchmark datasets across various domains show improved imputation and forecasting performance of NCDSSM over existing models. We evaluated NCDSSM on imputation and forecasting tasks on multiple benchmark datasets. |
| Researcher Affiliation | Collaboration | Abdul Fatir Ansari 1 Alvin Heng 2 Andre Lim 2 Harold Soh 2 3 1AWS AI Labs 2School of Computing, National University of Singapore (NUS) 3Smart Systems Institute, NUS. Correspondence to: Abdul Fatir Ansari <abdulfatir@u.nus.edu>. |
| Pseudocode | Yes | Algorithm 1 Learning in Neural Continuous-Discrete State Space Models Algorithm 2 Sum of Square Root Factors Algorithm 3 Continuous-Discrete Bayesian Filtering Algorithm 4 Continuous-Discrete Type II Extended RTS Smoothing |
| Open Source Code | Yes | Our code is available at https://github.com/clear-nus/NCDSSM. |
| Open Datasets | Yes | CMU Motion Capture (Walking) This dataset comprises walking sequences of subject 35 from the CMU Mo Cap database containing joint angles of subjects performing everyday activities. We used a preprocessed version of the dataset from Yildiz et al. (2019)... The original CMU Mo Cap database is available at: http://mocap.cs.cmu.edu. USHCN Climate Indicators... The preprocessed version of this dataset from De Brouwer et al. (2019) contains sporadic time series... The original USHCN Climate dataset is available at: https://cdiac.ess-dive.lbl.gov/ftp/ushcn daily/. Pymunk Physical Environments... used in Fraccaro et al. (2017)... |
| Dataset Splits | Yes | The training, validation, and test datasets consist of 5000, 500, and 500 sequences of length 30s each, respectively. (Bouncing Ball) The training, validation, and test datasets consist of 5000, 1000, and 1000 sequences of length 15s each, respectively. (Damped Pendulum) ...split into 16 training, 3 validation and 4 test sequences. (CMU Mo Cap) The 1,114 stations are split into 5 folds of 70% training, 20% validation, and 10% test stations, respectively. (USHCN Climate Indicators) Both datasets consist of 5000 training, 100 validation, and 1000 test videos with 60 frames each. (Pymunk Physical Environments) |
| Hardware Specification | Yes | We ran all our experiments on 2 machines with 1 Tesla T4 GPU, 16 CPUs, and 64 GB of memory each. |
| Software Dependencies | No | The paper mentions software components like 'Adam optimizer', 'ODE solver', 'Euler integrator', 'RK4 integrator' but does not specify their version numbers (e.g., Python version, PyTorch version, specific library versions). |
| Experiment Setup | Yes | We optimized all models using the Adam optimizer with a learning rate of 0.01 for all the datasets except Pymunk physical environments where we used 0.002. We reduced the learning rate exponentially with a decay rate of 0.9 every 500 steps for the bouncing ball, damped pendulum, and CMU Mo Cap (walking) datasets, every 100 steps for the USHCN climate dataset, and every 3000 steps for the Pymunk physical environments datasets. We trained the models for 5K, 2K, 2.5K, 150, and 100K steps with a batch size of 50, 64, 16, 100, and 32 for the bouncing ball, damped pendulum, CMU Mo Cap (walking), USHCN climate indicators, and Pymunk physical environments, respectively. |