Reinforcement Learning from Reachability Specifications: PAC Guarantees with Expected Conditional Distance
Authors: Jakub Svoboda, Suguman Bansal, Krishnendu Chatterjee
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We present (a) lower bound results which establish the necessity of ECD information for PAC guarantees and (b) an algorithm that establishes PAC-guarantees given the ECD information. To the best of our knowledge, this is the first RL from reachability specifications that does not make any assumptions about the underlying environment to learn policies with guarantees. |
| Researcher Affiliation | Academia | 1Institute of Science and Technology, Austria 2Georgia Institute of Technology, USA. |
| Pseudocode | Yes | Algorithm 1 Approximate MDP(S, S, ℓ, ε, p); Algorithm 2 Algorithm to find π Πε opt(M, T, ℓ).; Algorithm 3 Sum of v and v with given weights.; Algorithm 4 Weighted sum of two functions v and v .; Algorithm 5 PAC-MDP with ECD Learning Algorithm |
| Open Source Code | No | The paper does not contain any explicit statements or links indicating that source code for the described methodology is publicly available. |
| Open Datasets | No | The paper is theoretical and does not describe experiments with specific datasets or provide access information for any dataset. |
| Dataset Splits | No | The paper is theoretical and does not describe experiments with dataset splits for training, validation, or testing. |
| Hardware Specification | No | The paper is theoretical and focuses on algorithm design and PAC guarantees; it does not describe any experimental setup or specify hardware used. |
| Software Dependencies | No | The paper is theoretical and describes algorithms; it does not specify any software dependencies with version numbers. |
| Experiment Setup | No | The paper is theoretical and focuses on algorithm design and PAC guarantees; it does not describe any specific experimental setup details, hyperparameters, or training settings. |