Dimensionally Guided Synthesis of Mathematical Word Problems
Authors: Ke Wang, Zhendong Su
IJCAI 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We have realized our technique and extensively evaluated its efficiency and effectiveness. Results show that the system can generate hundreds of valid problems per second with varying levels of difficulty. More importantly, we show, via a user study with 30 students from a local middle school, that the generated problems are statistically indistinguishable from actual textbook problems for practice and examination. |
| Researcher Affiliation | Academia | Ke Wang Zhendong Su Department of Computer Science University of California, Davis {kbwang, su}@ucdavis.edu |
| Pseudocode | No | The paper describes the algorithm steps in paragraph form but does not include any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not mention releasing source code for the described methodology or provide any links. |
| Open Datasets | Yes | We selected 24 textbook problems from the Singapore Math curriculums [Publications, 2009a; 2009b; 2009c] and generated the same number of MWPs with an equivalent distribution of complexity. |
| Dataset Splits | No | The paper describes the selection of problems for the user study and how results were analyzed (e.g., partitioning textbook MWPs for comparison), but it does not specify explicit training, validation, or test splits for a model or dataset in a general sense that would be needed to reproduce the data partitioning for a machine learning experiment. |
| Hardware Specification | Yes | We have conducted our experiments on a desktop with a 4th generation Intel Core i7-4770 processor and 16GB RAM, running Ubuntu 12.04 LTS. |
| Software Dependencies | No | The paper mentions using 'Z3 [De Moura and Björner, 2008], an offthe-shelf state-of-the-art SMT solver,' but it does not specify a version number for Z3 or any other software dependencies. |
| Experiment Setup | Yes | We invited 30 seventh grade students from a local middle school to participate in the study. We selected 24 textbook problems from the Singapore Math curriculums [Publications, 2009a; 2009b; 2009c] and generated the same number of MWPs with an equivalent distribution of complexity. During the study, participants were given a single test containing all 48 problems, mixed randomly. The total duration for this pilot study was one and half hour, occupying two class periods. |