Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Playgol: Learning Programs Through Play
Authors: Andrew Cropper
IJCAI 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We experimentally test our claim on two domains: robot planning and real-world string transformations. Our experimental results suggest that playing can substantially improve learning performance. |
| Researcher Affiliation | Academia | Andrew Cropper University of Oxford EMAIL |
| Pseudocode | Yes | Algorithm 1 shows the Playgol algorithm |
| Open Source Code | No | The paper mentions that Playgol is implemented and uses Metagol (with a link to Metagol's GitHub), but it does not provide an explicit statement or link for the Playgol source code itself. |
| Open Datasets | Yes | Our dataset is based on the dataset from [Lin et al., 2014], which in turn is based on [Gulwani, 2011]. |
| Dataset Splits | No | The paper specifies 5 training examples and 5 testing examples per task in the string transformation experiment, but does not mention a separate validation split or explicit percentages for the splits. |
| Hardware Specification | No | The paper mentions a timeout of 60 seconds per task but does not provide any specific details about the hardware (e.g., CPU, GPU models, memory) used for the experiments. |
| Software Dependencies | No | The paper states 'Playgol uses Metagol [Cropper and Muggleton, 2016b]', but it does not specify a version number for Metagol or any other software dependencies. |
| Experiment Setup | Yes | For each p in {0, 200, 400, . . . , 2000}, we call playgol(Xn,BK,Tb,p,5) which returns a set of programs Pp. We enforce a timeout of 60 seconds per play and build task. |