BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration
Authors: Augustus Odena, Kensen Shi, David Bieber, Rishabh Singh, Charles Sutton, Hanjun Dai
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Overall, our empirical evaluation finds that the combination of learning and bottom-up search is remarkably effective, even with simple supervised learning approaches. We demonstrate the effectiveness of our technique on two datasets, one from the Sy Gu S competition and one of our own creation. |
| Researcher Affiliation | Industry | Augustus Odena , Kensen Shi , David Bieber, Rishabh Singh, Charles Sutton & Hanjun Dai Google Research {augustusodena,kshi,dbieber,rising,charlessutton,hadai}@google.com |
| Pseudocode | Yes | Algorithm 1 The BUSTLE Synthesis Algorithm |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code or a link to a code repository. |
| Open Datasets | No | The paper mentions evaluating on 'a new suite of 38 human-written benchmark tasks, which were designed to contain a variety of tasks difficult enough to stress our system' (of their own design, without public access info) and 'all Sy Gu S programs from the 2019 PBE SLIA TRACK and the 2018 PBE Strings Track' (without explicit public access link or citation for the dataset itself). It also mentions training on 'randomly generated synthetic data' which is created internally. |
| Dataset Splits | Yes | We first retain 10% of the training samples for validation, and identify the best number of training epochs. |
| Hardware Specification | Yes | our experiments are performed entirely on one CPU." and "On a single GPU with 16 GB of memory, the maximum beam size we can use is M = 80, 000 |
| Software Dependencies | No | The paper mentions 'our implementation is in Java' but does not provide specific version numbers for Java or any other software libraries or dependencies. |
| Experiment Setup | Yes | The new weight is computed from the discretized model output as w = w + 5 δ. This function is indicated by REWEIGHTWITHMODEL in Algorithm 1. ... discretize the model s output probability into an integer in δ {0, . . . , 5} by binning it into six bins bounded by [0.0, 0.1, 0.2, 0.3, 0.4, 0.6, 1.0]." and "We use a 3-layer LSTM with embedding size of 512. ... The batch size we use is 1024, with fixed learning rate 1e-3. |