Compositional Program Synthesis from Natural Language and Examples
Authors: Mohammad Raza, Sumit Gulwani, Natasa Milic-Frayling
IJCAI 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our approach on complex tasks from online help forums that are beyond the scope of current state-ofthe-art methods. We then present an evaluation of our technique on complex examples from online help forums and end with a discussion of related work and research outlook. The evaluation of our approach is based on string manipulation tasks from online help forums for Excel and regular expressions. Figure 5 provides a detailed table of experimental results comparing CPS against baselines. |
| Researcher Affiliation | Industry | Mohammad Raza Microsoft Research Cambridge, UK a-moraza@microsoft.com Sumit Gulwani Microsoft Research Redmond, USA sumitg@microsoft.com Natasa Milic-Frayling Microsoft Research Cambridge, UK natasamf@microsoft.com |
| Pseudocode | Yes | Figure 4: Program synthesis algorithm shows structured pseudocode for 'Synth Program' and 'Synth CSRStates' functions. |
| Open Source Code | No | The paper does not provide an explicit statement or a link to open-source code for the described methodology. |
| Open Datasets | No | The paper mentions using 'string manipulation tasks from online help forums for Excel and regular expressions' and provides general forum URLs (www.forums.devshed.com/regex-programming-147, www.stackoverflow.com, www.mrexcel.com). It also states 'Details of every task handled by our system can be found in [Raza et al., 2015]', which is a technical report. However, it does not provide concrete access (specific download link, DOI, or repository) to a structured public dataset. |
| Dataset Splits | No | The paper does not provide specific dataset split information (exact percentages, sample counts, or detailed splitting methodology) for training, validation, or testing. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'Stanford phrase structure parser' and 'SPLAT constituency parsers' but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | No | The paper mentions applying 'timeouts for rule application and recursive calls for component synthesis' but does not provide specific values for these or any other hyperparameters or detailed experimental setup configurations. |