Automatic Differentiation of Programs with Discrete Randomness
Authors: Gaurav Arya, Moritz Schauer, Frank Schäfer, Christopher Rackauckas
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We showcase how this method gives an unbiased and low-variance estimator which is as automated as traditional AD mechanisms. We demonstrate unbiased forward-mode AD of discrete-time Markov chains, agent-based models such as Conway s Game of Life, and unbiased reverse-mode AD of a particle filter. |
| Researcher Affiliation | Collaboration | Gaurav Arya Massachusetts Institute of Technology, USA aryag@mit.edu Moritz Schauer Chalmers University of Technology, Sweden University of Gothenburg, Sweden smoritz@chalmers.se Frank Schäfer Massachusetts Institute of Technology, USA University of Basel, Switzerland franksch@mit.edu Chris Rackauckas Massachusetts Institute of Technology, USA Julia Computing Inc., USA Pumas-AI Inc., USA crackauc@mit.edu |
| Pseudocode | Yes | Figure 3: Left: Stochastic triple structure (simplified). 1 struct Stochastic Triple 2 value # primal evaluation 3 δ # 'infinitesimal' component 4 Δs # component of discrete change 5 # with 'infinitesimal' 6 # probability |
| Open Source Code | Yes | Our code package is available at https://github.com/gaurav-arya/Stochastic AD.jl. |
| Open Datasets | No | The paper describes using a 'toy program', 'inhomogeneous random walk', 'stochastic Game of Life', and 'hidden Markov model' as the basis for its experiments. These are described as custom simulations or models rather than specific publicly available datasets with access information. |
| Dataset Splits | No | The paper does not explicitly provide information on dataset splits (training, validation, test) for its experiments. |
| Hardware Specification | No | We acknowledge the MIT Super Cloud and Lincoln Laboratory Supercomputing Center for providing HPC resources that have contributed to the research results reported within this paper. |
| Software Dependencies | No | We develop Stochastic AD.jl, a prototype package for stochastic AD based on our theory of stochastic derivatives. As discussed in Section 2.4, smoothed stochastic derivatives obey the usual chain rule, and thus can be used with existing AD infrastructure by supplying custom rules for discrete random constructs, and we do so for a particle filter in Section 3.4. ... exploiting Julia s multiple dispatch feature [33]. |
| Experiment Setup | No | The paper describes the specific parameters for the models used in experiments (e.g., random walk, Game of Life, particle filter with reference to Appendix C), but it does not specify general hyperparameter values or system-level training settings commonly found in machine learning experiments like learning rates, batch sizes, or optimizer configurations. |