LambdaBeam: Neural Program Search with Higher-Order Functions and Lambdas
Authors: Kensen Shi, Hanjun Dai, Wen-Ding Li, Kevin Ellis, Charles Sutton
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we experimentally evaluate the effectiveness of LAMBDABEAM, comparing to prior neural and symbolic approaches in an integer list manipulation domain. |
| Researcher Affiliation | Collaboration | Kensen Shi Google Deep Mind kshi@google.com Hanjun Dai Google Deep Mind hadai@google.com Wen-Ding Li Cornell University wl678@cornell.edu Kevin Ellis Cornell University kellis@cornell.edu Charles Sutton Google Deep Mind charlessutton@google.com |
| Pseudocode | No | The paper describes the MERGE operation and other algorithmic aspects but does not present them in a clearly labeled pseudocode or algorithm block. |
| Open Source Code | Yes | We release our LAMBDABEAM code and trained model checkpoints at https://github.com/ ellisk42/Lambda Beam. |
| Open Datasets | No | Similar to previous works including CROSSBEAM, we create synthetic training data by generating random tasks within our DSL. ... We furthermore removed from the training dataset all programs that would solve any of our 200 evaluation tasks, described below. |
| Dataset Splits | No | The paper describes generating synthetic training data and using handwritten and synthetically generated tasks for evaluation. It does not specify explicit train/validation/test splits with percentages or counts for reproducibility of data partitioning. |
| Hardware Specification | Yes | We trained the LAMBDABEAM model using on-policy training as in CROSSBEAM... which took about a week of training using 8 V100 GPUs. ... During evaluation, we use only 1 V100 GPU |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies such as programming languages, libraries, or frameworks used in their implementation. |
| Experiment Setup | Yes | In our experiments, we used the following hyperparameters for the LAMBDABEAM model architecture and training procedure. ... I/O Module: ...hidden size and output size of 512. Value Module: ...hidden size of 512 and output (embedding) size of 256... Search Context Summary Module: ...projection layer (denoted as MLPop in Figure 2, which projects back to the embedding dimension). Argument Selector Module: ...3-layer LSTM with hidden size of 256. The prediction head is a 2-layer MLP with hidden size of 512. During training, we generate on-policy data with beam size 10, use an effective batch size of 32, and use the Adam optimizer with a constant learning rate of 5 10 4. |