Fast Approximate Dynamic Programming for Infinite-Horizon Markov Decision Processes
Authors: Mohamad Amin Sharifi Kolarijani, Gyula Max, Peyman Mohajerin Mohajerin Esfahani
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4 Numerical simulations |
| Researcher Affiliation | Academia | M. A. S. Kolarijani Delft Center for Systems and Control Delft University of Technology The Netherlands M.A.Sharifi Kolarijani@tudelft.nl G. F. Max Delft Center for Systems and Control Delft University of Technology The Netherlands G.F.Max@tudelft.nl P. Mohajerin Esfahani Delft Center for Systems and Control Delft University of Technology The Netherlands P.Mohajerin Esfahani@tudelft.nl |
| Pseudocode | Yes | Algorithm 1 Conj VI: Approximate VI in conjugate domain |
| Open Source Code | Yes | We also provide the Conj VI MATLAB package [21] for the implementation of the proposed algorithm. The package also includes the numerical simulations of this section. and the reference '[21] M. A. S. Kolarijani and P. Mohajerin Esfahani. Conjugate value iteration (Conj VI) MATLAB package. Licensed under the MIT License, available online at https://github.com/Amin Kolarijani/ Conj VI, 2021.' |
| Open Datasets | No | The paper uses simulated control problems ('noisy inverted pendulum' and 'unstable deterministic batch reactor') based on setups borrowed from cited works [19, 20]. It does not involve using or providing access to publicly available pre-collected datasets in the conventional sense for training or evaluation. |
| Dataset Splits | No | The paper describes optimal control problems solved via simulation and does not refer to dataset splits for training, validation, or testing. |
| Hardware Specification | No | The paper does not provide specific hardware details such as CPU/GPU models, memory, or specific computing environments used for running the experiments. |
| Software Dependencies | No | The paper mentions the use of 'MATLAB package' and 'LLTd routine' but does not specify version numbers for these software dependencies, which would be necessary for reproducible replication. |
| Experiment Setup | Yes | The paper specifies experimental setup details such as the use of 'nearest neighbor extension' or 'multi-linear interpolation and extrapolation for the extension operators', 'dynamic scheme for the construction of Yg', and that 'all of the involved discrete sets are uniform grids with the same size N in each dimension', with specific values for N like 'N = 41' and 'N = 25'. |