Question Asking as Program Generation
Authors: Anselm Rothe, Brenden M. Lake, Todd Gureckis
NeurIPS 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our model using a data set containing natural language questions asked by human participants in an information-search game [19]. Given an ambiguous situation or context, our model can predict what questions human learners will ask by capturing constraints in how humans construct semantically meaningful questions. The method successfully predicts the frequencies of human questions given a game context, and can also synthesize novel human-like questions that were not present in the training set. |
| Researcher Affiliation | Academia | Anselm Rothe1 anselm@nyu.edu Brenden M. Lake1,2 brenden@nyu.edu Todd M. Gureckis1 todd.gureckis@nyu.edu 1Department of Psychology 2Center for Data Science New York University |
| Pseudocode | No | The paper describes a grammar and rules for program generation but does not include a clearly labeled "Pseudocode" or "Algorithm" block. |
| Open Source Code | No | The paper provides a link (https://github.com/anselmrothe/question_dataset) which is explicitly described as containing the "question data set", not the source code for the model or methodology. |
| Open Datasets | Yes | Our analysis on a data set we collected in [19], which consists of 605 natural language questions asked by 40 human players to resolve an ambiguous game situation (similar to Battleship ).1 1https://github.com/anselmrothe/question_dataset |
| Dataset Splits | Yes | To make predictions, the different candidate models were fit to 15 contexts and asked to predict the remaining one (i.e., leave one out cross-validation). |
| Hardware Specification | No | The paper does not specify any hardware details (e.g., CPU, GPU models, memory) used for the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., Python, PyTorch, specific libraries and their versions). |
| Experiment Setup | Yes | To run the procedure for a given model and training set, we ran 100,000 iterations of gradient ascent at a learning rate of 0.1. |