Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Online Optimization for Offline Safe Reinforcement Learning
Authors: Yassine Chemingui, Aryan Deshwal, Alan Fern, Thanh Nguyen-Tang, Jana Doppa
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results on the DSRL benchmark demonstrate that our method reliably enforces safety constraints under stringent cost budgets, while achieving high rewards. The code is available at https://github.com/yassine Ch/O3SRL. 1 Introduction Offline reinforcement learning (RL) [Levine et al., 2020] is a powerful paradigm to learn decisionmaking policies from logged datasets without the need for additional interaction with the environment. Offline RL has shown good success in domains including autonomous driving, robotics, control systems, and black-box optimization [Fang et al., 2022, Lin et al., 2024, Li et al., 2024, Zhan et al., 2022, Chemingui et al., 2024] where executing exploratory actions is not practical. |
| Researcher Affiliation | Academia | Yassine Chemingui Washington State University EMAIL Aryan Deshwal University of Minnesota EMAIL Alan Fern Oregon State University EMAIL Thanh Nguyen-Tang New Jersey Institute of Technology EMAIL Janardhan Rao Doppa Washington State University EMAIL |
| Pseudocode | Yes | Algorithm 1 provides a pseudo-code of the O3SRL framework. ... Algorithm 2 O3SRL Approximation via Stochastic Oracle and EXP3 Multi-Arm Bandit Strategy |
| Open Source Code | Yes | Empirical results on the DSRL benchmark demonstrate that our method reliably enforces safety constraints under stringent cost budgets, while achieving high rewards. The code is available at https://github.com/yassine Ch/O3SRL. |
| Open Datasets | Yes | We evaluate O3SRL and baseline methods using the DSRL Bullet benchmark [Liu et al., 2024], which provides standardized offline datasets for safe RL research and evaluation. |
| Dataset Splits | No | The paper does not explicitly provide specific dataset split information (e.g., percentages, sample counts for training, validation, and test sets) within the main text or appendix. It mentions evaluation metrics averaged over '20 evaluation episodes, and three random seeds', but this is about evaluation methodology rather than dataset partitioning. |
| Hardware Specification | Yes | All experiments were conducted using an NVIDIA A40 GPU with 48GB of memory. |
| Software Dependencies | No | The paper mentions software components like 'Optimizer Adam' and uses algorithms like 'TD3+BC' and 'IQL' but does not provide specific version numbers for these or other key software dependencies (e.g., Python, PyTorch, CUDA libraries). |
| Experiment Setup | Yes | Table 11: O3SRL Hyperparameters EXP3 parameters Number of arms k 5 Maximum C 5 λ update frequency M 10 λ learning rate η 2e-3 TD3BC parameters Discount γ 0.99 Policy noise 0.2 Policy noise clip (0.5, 0.5) Policy update frequency 2 α 2.5 Optimizer Adam Actor, Critic learning rate 3e-4 Actor, Critic hidden size 256 Training steps 100000 Seed [10, 20, 30] |