Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Identifiability in inverse reinforcement learning
Authors: Haoyang Cao, Samuel Cohen, Lukasz Szpruch
NeurIPS 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The code and instructions was included in the supplemental material. We did not work with any particular dataset. ... For the purpose of demonstration, instead of reporting error bars, we included training and test errors with respect to each random seed. ... All the experiments have been conducted on Google Research Colaboratory with GPU as hardware accelerator. The experiment in Section 4 took 2 minutes and 14 seconds in total for all 6 random initializations. |
| Researcher Affiliation | Academia | Haoyang Cao Alan Turing Institute EMAIL Samuel N. Cohen Mathematical Institute, University of Oxford and Alan Turing Institute EMAIL Łukasz Szpruch School of Mathematics, University of Edinburgh and Alan Turing Institute EMAIL |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code and instructions was included in the supplemental material. |
| Open Datasets | No | We did not work with any particular dataset. |
| Dataset Splits | No | The paper states, "We did not work with any particular dataset," which implies no dataset splits, including validation splits, were used for the experiments described. |
| Hardware Specification | No | All the experiments have been conducted on Google Research Colaboratory with GPU as hardware accelerator. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers. |
| Experiment Setup | No | The ethics statement claims "We specified training details in the main text" and "The experiment in Section 4 took 2 minutes and 14 seconds in total for all 6 random initializations." However, the provided main text of the paper does not contain specific experimental setup details, hyperparameters, or training configurations for these numerical demonstrations. |