Modeling Irregular Time Series with Continuous Recurrent Units
Authors: Mona Schirmer, Mazin Eltayeb, Stefan Lessmann, Maja Rudolph
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically study the CRU on a number of challenging datasets and find that it can interpolate irregular time series better than methods based on neural ordinary differential equations. |
| Researcher Affiliation | Collaboration | 1Humboldt-Universit at zu Berlin, Germany. Work done during an internship at Bosch Center for AI 2Bosch Center for AI, Germany 3Humboldt-Universit at zu Berlin, Germany 4Bosch Center for AI, USA. |
| Pseudocode | Yes | Algorithm 1 The CRU Input: Datapoints and their timestamps {(xt, t)}t T Initialize: µ+ t0 = 0, Σ+ t0 = 10 I for observation times t > t0 T do yt, σobs t = fθ(xt) µ t , Σ t = predict(µ+ τ(t), Σ+ τ(t), t τ(t)) µ+ t , Σ+ t = update(µ t , Σ t , yt, σobs t ) ot, σout t = gϕ(µ+ t , Σ+ t ) end for Return: {ot, σout t }t T |
| Open Source Code | Yes | A PyTorch implementation is available on github.1 |
| Open Datasets | Yes | The United States Historical Climatology Network (USHCN) dataset (Menne et al., 2015) contains daily measurements... (Section 4.1) and Daily weather records can be downloaded at https://cdiac.ess-dive.lbl.gov/ftp/ushcn_daily/. (Appendix E.1). Also, Finally, we also benchmark the models on the data set of the Physionet Computing in Cardiology Challenge 2012 (Silva et al., 2012). (Section 4.1) and The data is publicly available for download at https://physionet.org/content/challenge-2012/1.0.0/. (Appendix E.2). |
| Dataset Splits | Yes | We used 2 000 sequences for training and 1 000 for validation and testing each. (Section 4.1, Pendulum) and We test models on a 20% hold-out set and trained on 80% of which we used 25% for validation. (Section 4.1, USHCN) and We split the data into 20% test and 80% train set of which we used 25% for validation. (Section 4.1, Physionet) |
| Hardware Specification | Yes | Models were trained on one Nivida TU102GL Quatro RTX 6000/8000 with 40 physical Intel Xeon Gold 6242R CPU. |
| Software Dependencies | No | The paper mentions 'PyTorch implementation' and 'Adam optimizer', but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | Training In all experiments, we train each model for 100 epochs using the Adam optimizer (Kingma & Ba, 2015). Reported MSE and Gaussian NLL results are averages over 5 runs. We used a batch size of 50 for the pendulum and USHCN data and a batch size of 100 for Physionet. For USHCN and Physionet, we split the data into 80% train and 20% test and used 25% of the train set for validation. (Appendix D.8) and On the validation set in the Pendulum interpolation experiment, we found a learning rate of 0.001 to work best for CRU and a slightly higher learning rate of 0.005 for f-CRU. (Appendix D.8) |