Synthesizing Pattern Programs from Examples
Authors: Sunbeom So, Hakjoo Oh
IJCAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We implemented the algorithm in a tool and evaluated it on 40 exercises gathered from online forums. The experimental results and user study show that our tool can synthesize instructive solutions from 1 3 example patterns in 1.2 seconds on average. 5 Evaluation Experimental setup To demonstrate the effectiveness of our approach, we collected 40 benchmarks from several online forums. These problems are comprised of various geometric patterns that are helpful for students learning, including challenging ones. All of the experiments were conducted on Mac Book Pro with Intel Core i7 and 16GB of memory. Algorithm performance The experimental results show that our synthesis algorithm is remarkably efficient; PAT solves the benchmarks 1.2 seconds on average, only from 1 3 examples (the column Final and Ex in Table 1). ... User study We also evaluated PAT with 23 undergraduates... |
| Researcher Affiliation | Academia | Sunbeom So Korea University sunbeom so@korea.ac.kr Hakjoo Oh Korea University hakjoo oh@korea.ac.kr |
| Pseudocode | Yes | Algorithm 1 Synthesis Algorithm Input: A set of examples E Output: A program consistent with E 1: W {s0} where s0 = 2: Γ a candidate set of integer triplets 3: repeat 4: Pick the smallest state s from W 5: if s is a terminal state then 6: if solution(s) then return for i in N do (s; ) 7: else 8: W W next(s) 9: until W = |
| Open Source Code | Yes | We make our tool and data publicly available.1 1http://prl.korea.ac.kr/pat |
| Open Datasets | No | No concrete access information (link, DOI, formal citation with authors/year) for a publicly available dataset is provided. The paper mentions "40 benchmarks gathered from online forums" but does not provide details on how to access these specific benchmarks. |
| Dataset Splits | No | No specific dataset split information (exact percentages, sample counts, citations to predefined splits, or detailed splitting methodology) is provided. The paper mentions using '1-3 example patterns' for synthesis, which refers to the input for their system, not a train/validation/test split of a dataset. |
| Hardware Specification | Yes | All of the experiments were conducted on Mac Book Pro with Intel Core i7 and 16GB of memory. |
| Software Dependencies | No | No specific ancillary software details with version numbers are provided. The paper mentions using an 'off-the-shelf SMT solver' but does not name it or specify its version. No other software dependencies with versions are listed. |
| Experiment Setup | No | No specific experimental setup details such as concrete hyperparameter values, training configurations, or system-level settings (e.g., learning rates, batch sizes, epochs) are provided. The paper describes the algorithmic components but does not detail tunable parameters in the manner typical for machine learning experiments. |