Variational Inference for SDEs Driven by Fractional Noise
Authors: Rembert Daems, Manfred Opper, Guillaume Crevecoeur, Tolga Birdal
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 EXPERIMENTS We implemented our method in JAX (Bradbury et al., 2018), using Diffrax (Kidger, 2021) for SDE solvers, Optax (Babuschkin et al., 2020) for optimization, Diffrax (Babuschkin et al., 2020) for distributions and Flax (Heek et al., 2023) for neural networks. [...] We train models on Stochastic Moving MNIST (Denton & Fergus, 2018), a video dataset where two MNIST numbers move on a canvas and bounce off the edge with random velocity in a random direction. Our MA-f BM driven model is on par with closely related discrete-time methods such as SVG (Denton & Fergus, 2018) or SLRVP Franceschi et al. (2020), in terms of PSNR, and is better than the BM baseline in terms of PSNR and ELBO (Tab. 1). |
| Researcher Affiliation | Academia | Rembert Daems 1,2 Manfred Opper 3,4,5 Guillaume Crevecoeur 1,2 Tolga Birdal 6 1 D2LAB, Ghent University, Belgium 2 MIRO core lab, Flanders Make@UGent, Belgium 3 Dept. of Theor. Comp. Science, Technical University of Berlin, Germany 4 Inst. of Mathematics, University of Potsdam, Germany 5 Centre for Systems Modelling and Quant. Biomed., University of Birmingham, UK 6 Dept. of Computing, Imperial College London, UK |
| Pseudocode | No | The paper does not contain any sections or blocks explicitly labeled 'Pseudocode' or 'Algorithm'. |
| Open Source Code | Yes | We make our implementation publicly available under: github.com/Video Neural SDE/MAFBM. |
| Open Datasets | Yes | We train models on Stochastic Moving MNIST (Denton & Fergus, 2018), a video dataset... We also report results on a real-world video dataset of a double pendulum (Asseman et al., 2018)... |
| Dataset Splits | Yes | We train on sequences of 25 frames, with a time length of 2.4 (0.1 per frame). [...] We evaluate the stochastic video predictions by sampling 100 predictions and reporting the Peak Signal-to-Noise Ratio (PSNR) of the best sample, calculated frame-wise and averaged over time. [...] Furthermore, we report the ELBO on the test set, indicating how well the model has captured the data. [...] We use the train-test split from the original dataset (Asseman et al., 2018). |
| Hardware Specification | Yes | Models were trained on a single NVIDIA GeForce RTX 4090, which takes around 39 hours for one model. |
| Software Dependencies | Yes | We implemented our method in JAX (Bradbury et al., 2018), using Diffrax (Kidger, 2021) for SDE solvers, Optax (Babuschkin et al., 2020) for optimization, Diffrax (Babuschkin et al., 2020) for distributions and Flax (Heek et al., 2023) for neural networks. |
| Experiment Setup | Yes | Models are trained for 2000 training steps with a batch size of 32. We use the Adam (Kingma & Ba, 2014) optimizer with fixed learning rate 10 3. We use the Stratonovich Milstein SDE solver (Kidger, 2021) with an integration step of 0.01. The length of the bridge T = 2 and observation noise σ = 0.1. [...] Each model was trained for 187500 training steps with a batch size of 32. We use the Adam (Kingma & Ba, 2014) optimizer with fixed learning rate 3 10 4. We use the Stratonovich Milstein SDE solver (Kidger, 2021) with an integration step of 0.033 (3 integration steps per data frame). |