Neural Guided Constraint Logic Programming for Program Synthesis
Authors: Lisa Zhang, Gregory Rosenblatt, Ethan Fetaya, Renjie Liao, William Byrd, Matthew Might, Raquel Urtasun, Richard Zemel
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments focus on synthesizing programs in a subset of Lisp, and show that scoring constraints help. More importantly, we test the generalizability of our approach on three families of synthesis problems. We compare against state-of-the-art systems λ2 [11], Escher [12], Myth [13], and Robust Fill [4]. We show that our approach has the potential to generalize to larger problems. |
| Researcher Affiliation | Collaboration | Lisa Zhang1,2, Gregory Rosenblatt4, Ethan Fetaya1,2, Renjie Liao1,2,3, William E. Byrd4, Matthew Might4, Raquel Urtasun1,2,3, Richard Zemel1,2 1University of Toronto, 2Vector Institute, 3Uber ATG, 4University of Alabama at Birmingham |
| Pseudocode | No | The paper describes algorithmic steps in prose but does not include any formally labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | We contribute a modified mini Kanren, drivable by an external agent, available at https://github.com/xuexue/neuralkanren. |
| Open Datasets | No | The paper states 'We programmatically generate training data by querying (evalo P I O) in mini Kanren', describing the generation process rather than providing concrete access (link, DOI, formal citation) to a pre-existing public dataset. |
| Dataset Splits | No | The paper mentions '500 generated problems for training' and '100 generated test problems', but does not explicitly describe a validation dataset split or a cross-validation strategy. |
| Hardware Specification | Yes | All test experiments are run on Intel i7-6700 3.40GHz CPU with 16GB RAM. |
| Software Dependencies | No | The paper mentions implementing a 'Python interface' and mini Kanren being implemented in 'Scheme', but does not provide specific version numbers for these or any other key software libraries or frameworks used in the experiments (e.g., PyTorch, TensorFlow, specific Python libraries). |
| Experiment Setup | Yes | The RNN model uses 2-layer bi-directional LSTMs with embedding size of 128. The GNN model uses a single up/down/up pass with embedding size 64 and message size 128. Increasing the number of passes did not yield improvements. Further, we compare against a baseline RNN model that does not take constraints as input: instead, it computes embeddings of the input, output, and the candidate partial program using an LSTM, then scores the concatenated embeddings using a MLP. This baseline model also uses 2-layer bi-directional LSTMs with embedding size of 128. All models use a 2-layer neural network with Re LU activation as the scoring function. For optimization we use RMSProp [30], with weight decay for regularization. |