Efficient and Accurate Gradients for Neural SDEs
Authors: Patrick Kidger, James Foster, Xuechen (Chen) Li, Terry Lyons
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate the empirical performance of the reversible Heun method. For space, we present abbreviated details and results here. See Appendix F for details of the hyperparameter optimisation procedure, test metric definitions, and so on, and for further results on additional datasets and additional metrics. Versus midpoint We begin by comparing the reversible Heun method with the midpoint method, which also converges to the Stratonovich solution. We train an SDE-GAN on a dataset of weight trajectories evolving under stochastic gradient descent, and train a Latent SDE on a dataset of air quality over Beijing. See Table 1. |
| Researcher Affiliation | Academia | Patrick Kidger1 James Foster1 Xuechen Li2 Terry Lyons1 1 University of Oxford; The Alan Turing Insitute 2 Stanford {kidger, foster, tlyons}@maths.ox.ac.uk lxuechen@cs.toronto.edu |
| Pseudocode | Yes | Algorithm 1: Forward pass Input: tn, zn, bzn, µn, σn, t, W ... Algorithm 2: Backward pass Input: tn+1, zn+1, bzn+1, µn+1, σn+1, t, W, zn+1 , L(ZT ) ... Algorithm 3: Sampling the Brownian Interval Type: Let Node denote a 5-tuple consisting of an interval, a seed, and three optional Nodes, corresponding to the parent node, and two child nodes, respectively. |
| Open Source Code | Yes | We have contributed implementations of all of our techniques to the torchsde library to help facilitate their adoption. To facilitate the use of the techniques introduced here in particular without requiring a technical background in numerical SDEs we have contributed implementations of both the reversible Heun method and the Brownian Interval to the open-source torchsde [42] package. |
| Open Datasets | Yes | We train an SDE-GAN on a dataset of weight trajectories evolving under stochastic gradient descent, and train a Latent SDE on a dataset of air quality over Beijing. ... [85] D. Dua and C. Graff. UCI Machine Learning Repository, 2017. URL http://archive.ics. uci.edu/ml. [86] S. Zhang, B. Guo, A. Dong, J. He, Z. Xu, and S. X. Chen. Cautionary tales on air-quality improvement in Beijing. Proceedings of the Royal Society A, 473(2205), 2017. |
| Dataset Splits | Yes | Full training details (including data splits, hyperparameter optimisation, and so on) are available in Appendix F. |
| Hardware Specification | No | The paper states, "Compute resources and computed time are specified in Appendix F." However, the main body of the paper does not contain explicit hardware specifications such as specific GPU or CPU models. |
| Software Dependencies | No | The paper mentions 'torchsde library' and 'PyTorch [27]' but does not provide specific version numbers for these or any other software dependencies in the main text. |
| Experiment Setup | No | The paper states, "See Appendix F for details of the hyperparameter optimisation procedure, test metric definitions, and so on, and for further results on additional datasets and additional metrics." The main text does not include specific hyperparameter values or detailed system-level training settings. |