Stochastic Optimal Control and Estimation with Multiplicative and Internal Noise
Authors: Francesco Damiani, Akiyuki Anzai, Jan Drugowitsch, Gregory DeAngelis, Ruben Moreno Bote
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We apply our algorithm and compare it with the state-of-the-art solutions in two scenarios governed by a linear dynamical system (Eqs. 5-8). Hereafter, GD refers to our numerical algorithm (Section 3.1), and TOD refers to the algorithm from [1]. First, in a simplified one-dimensional reaching task (m = p = k = 1) with all noise sources present, we show that for non-zero internal noise, Ωη > 0, GD outperforms TOD, resulting in a lower accumulated cost. Second, in a reaching task with a four-dimensional state and one-dimensional control and sensory feedback (m = 4, p = k = 1), GD predicts qualitatively different behavior and shows a 90% performance improvement when internal noise contributes 10% of the total. |
| Researcher Affiliation | Academia | Francesco Damiani Center for Brain and Cognition, Department of Engineering Pompeu Fabra University Barcelona, ES francesco.damiani@upf.edu Akiyuki Anzai Department of Brain and Cognitive Sciences University of Rochester Rochester, USA aanzai@ur.rochester.edu Jan Drugowitsch Department of Neurobiology Harvard Medical School Boston, USA jan_drugowitsch@hms.harvard.edu Gregory C. De Angelis Department of Brain and Cognitive Sciences University of Rochester Rochester, USA gdeangelis@ur.rochester.edu Rubén Moreno-Bote Center for Brain and Cognition, Department of Engineering, Serra Húnter Fellow Programme Pompeu Fabra University Barcelona, ES ruben.moreno@upf.edu |
| Pseudocode | Yes | Algorithm 1 Propagation of the expected cost GD algorithm (Appendix A.4.2) and Algorithm 2 FPOMP algorithm (Appendix A.6.3). |
| Open Source Code | Yes | The codes to generate the data and the figures are provided in the supplemental material. (from NeurIPS Paper Checklist, Question 5 Justification) |
| Open Datasets | No | The paper defines system parameters and initial conditions (e.g., Table 2, Table 3) for its simulations, which are used to generate data for its experiments (e.g., "50k trials"). It does not use or provide access information for a pre-existing publicly available dataset. |
| Dataset Splits | No | The paper runs Monte Carlo simulations (e.g., "50k trials") and discusses the training of models (e.g., "We trained our models using..."). However, it does not explicitly define training, validation, or test dataset splits with percentages, sample counts, or specific predefined splits for its experimental setup. |
| Hardware Specification | No | However, for the multi-dimensional sensorimotor task presented above, the GD algorithm takes significantly longer: while the TOD algorithm completes in just a few minutes on a standard laptop, the GD optimization requires several hours (approximately 4 hours). (This mentions a "standard laptop" but lacks specific hardware details like CPU/GPU models or memory.) |
| Software Dependencies | No | For the GD algorithm (Section 3.1) we minimize the expected accumulated cost E[J], computed through Algorithm 1, using the function "Gradient Descent()" in the "Optim.jl" Julia package. (No version numbers provided for Julia or Optim.jl). |
| Experiment Setup | Yes | The system parameters are provided in Table 2 in Appendix A.5.1 (note that we define the strength of the internal noise as ση = pΩη). The parameters of the problem are listed in Table 3. Table 1: Hyper-parameters of the used algorithms. (Tables 2 and 3 list numerical parameters like "t time-step (s) 0.010", "r Control-dependent cost at each t < T 1e 5", "T time steps 100", "ση std of the additive internal Gaussian noise ηt {0.0 : 0.1 : 2.0}"). |