Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Provably Correct Optimization and Exploration with Non-linear Policies
Authors: Fei Feng, Wotao Yin, Alekh Agarwal, Lin Yang
ICML 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically evaluate this adaptation, and show that it outperforms prior heuristics inspired by linear methods, establishing the value in correctly reasoning about the agent s uncertainty under non-linear function approximation. We conduct experiments to testify the effectiveness of ENIAC. |
| Researcher Affiliation | Collaboration | 1Department of Mathematics, University of California, Los Angeles, Los Angeles, CA, USA. 2Microsoft Research, Redmond, WA, USA. 3Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, USA. |
| Pseudocode | Yes | Algorithm 1 Exploratory Non-Linear Incremental Actor Critic (ENIAC); Algorithm 2 Policy Update |
| Open Source Code | Yes | Check our code at https://github.com/FlorenceFeng/ENIAC. |
| Open Datasets | Yes | We test on a continuous control task which requires exploration: continuous control Mountain Car5 from Open AI Gym (Brockman et al., 2016). 5https://gym.openai.com/envs/Mountain Car Continuous-v0/ |
| Dataset Splits | No | The paper mentions evaluating methods over "10 random seeds" and varying "depths of networks" but does not specify any training/test/validation dataset splits (e.g., percentages or counts). |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions using PPO and fully-connected neural networks (FCNN) and refers to PyTorch in a citation, but it does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | We evaluate all methods on varying depths of networks: 2-layer stands for (64, 64) hidden units, 4-layer for (64, 128, 128, 64), and 6-layer for (64, 64, 128, 128, 64, 64). Layers are connected with ReLU non-linearities. Hyperparameters for all methods are provided in Appendix F. |