Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Goal Recognition for Deceptive Human Agents through Planning and Gaze
Authors: Thao Le, Ronal Singh, Tim Miller
JAIR 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluated our models in two human-subject studies: (1) using data collected from 30 individuals playing a navigation game inspired by an existing deception study and (2) using data collected from 40 individuals playing a competitive game (Ticket To Ride). We found that one of our models (Modulated Deception Gaze+Ontic) offers promising results compared to the previous state-of-the-art model in both studies. |
| Researcher Affiliation | Academia | Thao Le EMAIL Ronal Singh EMAIL Tim Miller EMAIL School of Computing and Information Systems The University of Melbourne, Parkville VIC 3010, Australia |
| Pseudocode | No | The paper describes its models and methods using mathematical equations and prose (e.g., Equations 1-21) but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The game is designed in Java Script language and can be played in a web browser1. 1. https://github.com/thaole25/vip-routing |
| Open Datasets | Yes | The data was collected and used by Newn et al. (2018) and Singh et al. (2020). |
| Dataset Splits | No | The paper mentions data collection details like the number of participants and games (e.g., "30 (participants) × 10 (games) = 300 (games) in each condition"), but it does not specify how these datasets were split into training, validation, or test sets for model evaluation or reproduction. |
| Hardware Specification | Yes | The computer is fitted with a Tobii Pro X3-120 eye tracker (120Hz). |
| Software Dependencies | No | The paper states: "The game is designed in Java Script language" and mentions "Tobii Studio software". However, it does not provide specific version numbers for Java Script, Tobii Studio, or any other libraries or frameworks used. |
| Experiment Setup | Yes | In our experiments, we chose Duration Threshold = 350 (ms) based on prior work on normal human fixation duration (Qvarfordt, 2017) and α = 0.05... we choose C = π 80/180 ≈ 1.4 (radians)... we chose Distance Threshold = 150 (pixels) (39.69 mm), and α = 0.05... In our study, we choose X = 2(s). |