Free-Form Variational Inference for Gaussian Process State-Space Models
Authors: Xuhui Fan, Edwin V. Bonilla, Terence O’Kane, Scott A Sisson
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our FFVD method on synthetic data and on six real-world system identification benchmarks (Ialongo et al., 2019; Doerr et al., 2018), comparing it with VGPSSM (Frigola et al., 2014), PRSSM (Doerr et al., 2018), VCDT (Ialongo et al., 2019), and use a LSTM network (Hochreiter & Schmidhuber, 1997) as a baseline non-GP based model. The test performance is shown in Table 2 and Table 3, where we see that FFVD attains the best NMLL values in three out of the six benchmarks and obtains lowest RMSE values in four out of the six benchmarks (in bold). |
| Researcher Affiliation | Academia | 1University of Newcastle, Australia 2CSIRO s Data61, Australia 3CSIRO s Environment, Australia 4University of New South Wales, Australia. |
| Pseudocode | Yes | Algorithm 1 Particle MCMC for inferring latent states x0:T |
| Open Source Code | Yes | Our code and supplementary material can be found at https://github.com/xuhuifan/FFVD. As mentioned in the main paper, our code is publicly available at https://github.com/xuhuifan/FFVD. |
| Open Datasets | Yes | We evaluate our FFVD method on synthetic data and on six real-world system identification benchmarks (Ialongo et al., 2019; Doerr et al., 2018). Regarding the benchmarks, we have 1 024 observations for Actuator, 1 000 observations for Ballbeam, 500 observations for Drive, 1 000 observations for Dryer, 1 024 for Flutter, and 296 observations for Furnace. |
| Dataset Splits | No | The paper only specifies training and test lengths ("Training lengths are: Ttrain = 500, 500, 250, 500, 500, 150, respectively, and test lengths are the rest of the sequences.") but does not explicitly provide details for a separate validation split or how it was used to reproduce the experiment. |
| Hardware Specification | Yes | These time results were done using a Macbook Pro 2021 with 16GB in memory, M1 chip, and 8 cores. |
| Software Dependencies | No | The paper mentions "scipy.stats.normaltest from Python s scipy package" and "Adam" optimizer, but does not provide specific version numbers for Python, scipy, or any other key software libraries or dependencies used in the experiments. |
| Experiment Setup | Yes | For optimizing these hyper-parameters, we use Adam, with default settings for the optimizer parameters except for a decayed learning rate. We run our FFVD algorithm for 50 000 iterations and VCDT for 200 000. We set the learning rate as 0.01 and the decay parameter as 0.05 during SGHMC sampling. We used S = 105 posterior samples for VCDT (as in the results in the original paper) and S = 100 for ours. The number of inducing points is set to 20, and these 20 inducing points are evenly spread in the interval [ 2, 2]. We set the length of the training trajectory as 120, the number of iterations as 50 000, and the number of posterior samples as 50. We set the number of inducing points to M = 100 and the number of dimensions for latent states x0:T to dx = 4. |