Generating Content for Scenario-Based Serious-Games Using CrowdSourcing
Authors: Sigal Sina, Avi Rosenfeld, Sarit Kraus
AAAI 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluated Scenario Gen in 6 different content domains and found that it was consistently rated as coherent and consistent as the originally captured content. We also compared Scenario Gen s content to that created by traditional planning techniques. We found that both methods were equally effective in generating coherent and consistent scenarios, yet Scenario Gen s content was found to be more varied and easier to create. |
| Researcher Affiliation | Academia | Sigal Sina Bar-Ilan University, Israel sinasi@macs.biu.ac.il Avi Rosenfeld Jerusalem College of Technology, Israel rosenfa@jct.ac.il Sarit Kraus Bar-Ilan University, Israel sarit@cs.biu.ac.il |
| Pseudocode | Yes | Algorithm 1 Scenarios Generator (Scenario Gen) Require: User profile P, original scenario O Scn and a constraints set S Ensure: Revise scenario N Scn 1: Run Max Sat solver to get a new scenario N Scn with AI placeholders 2: while N Scn includes placeholders do 3: Run KAR to replace a placeholder AI 4: Run SNACS to extend the chosen replacement AI into ADR 5: Run Max Sat solver to validate N Scn 6: end while 7: return N Scn |
| Open Source Code | No | The paper does not provide a direct link or explicit statement about releasing the source code for the described methodology. |
| Open Datasets | No | The paper describes creating its own datasets (DSA, DSD) using Amazon Mechanical Turk workers. While it provides a link to questionnaires used to crowdsource the data (http://aimamt.azurewebsites.net/), it does not provide access to the collected datasets themselves or state that they are publicly available. |
| Dataset Splits | No | The paper describes using collected daily schedules as 'original scenarios' for evaluation and randomly cutting sections for replacement ('We randomly cut a section of 7-8 hours from each of the original activities list...'). However, it does not specify explicit training, validation, or test dataset splits or percentages required for reproducibility of model training. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts, or cloud instance types) used for running experiments. |
| Software Dependencies | No | The paper mentions 'akmaxsat solver (Kuegel 2010)', 'Simple NLG (Gatt and Reiter 2009)', and 'SHOP2 (Nau et al. 2003)' as tools used. While these are specific tools, the paper does not provide explicit version numbers for all key software components or libraries necessary for exact replication, nor does it specify exact version numbers for the named solvers. |
| Experiment Setup | Yes | To check the integrity of the scenario s activities list after the replacement of one of its activities, we used the daily schedules we already collected from the crowd for DSA, as the original scenario... We randomly cut a section of 7-8 hours from each of the original activities list... We associate different weights for each attribute with a scoring function that gets an attribute and a similarity measure and returns a score within the range [-15,15]. We refine this score function using several preliminary trial and error iterations. |