Evaluating the Interpretability of the Knowledge Compilation Map: Communicating Logical Statements Effectively
Authors: Serena Booth, Christian Muise, Julie Shah
IJCAI 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct a user study to evaluate whether knowledge compilation can aid logic interpretability. We find only sparse effects of knowledge compilation properties on interpretability. We discover some languages considered to be compilation-only are acceptable, while disjunctive normal form a representation assumed to be interpretable is not significantly more interpretable than other forms. |
| Researcher Affiliation | Collaboration | Serena Booth1 , Christian Muise2,3 and Julie Shah1 1MIT Computer Science and Artiļ¬cial Intelligence Laboratory 2IBM Research 3MIT-IBM Watson AI Lab {serenabooth, julie a shah}@csail.mit.edu, christian.muise@ibm.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our study procedures and source code are available at github.com/serenabooth/logic-interpretability. |
| Open Datasets | No | The paper describes generating data through simulations ('We simulate the agents and generate traces...') but does not provide concrete access information (link, DOI, repository, or formal citation to a public dataset) for this data or specify the use of a well-known public dataset. |
| Dataset Splits | No | The paper describes a user study and scenario questions but does not provide specific dataset split information (percentages, sample counts, or citations to predefined splits) for training, validation, or testing. |
| Hardware Specification | No | The paper mentions participants completed the study 'on-site' but does not provide specific hardware details (e.g., exact GPU/CPU models, memory amounts, or detailed computer specifications) used for running its experiments or simulations. |
| Software Dependencies | No | The paper mentions software tools like 'PMC PREPROCESSOR' and 'DSHARP' but does not provide specific version numbers for these or any other ancillary software components needed to replicate the experiment. |
| Experiment Setup | No | The paper describes the setup of a user study (e.g., number of questions, presentation format) but does not provide specific experimental setup details such as hyperparameter values, training configurations, or system-level settings for any computational models. |